feat: add per-turn reasoning effort selection to chat API - #26975
Conversation
Docs preview📖 View docs preview for |
394a845 to
e26d472
Compare
8e1bc4e to
4968c7e
Compare
|
/coder-agents-review |
|
Chat: Review posted | View chat Review historydeep-review v0.9.0 | Round 3 | Last posted: Round 3, 11 findings (1 P2, 5 P3, 1 P4, 4 Nit), APPROVE. Review Finding inventoryFindings
Contested and acknowledgedCRF-4 (P3, exp_chats.go:1241) - Validation error does not echo the rejected value
CRF-7 (Nit, exp_chats.go:1242) - Hardcoded effort list in error detail
CRF-8 (Nit, exp_chats.go:1238) - chatReasoningEffort naming inconsistency
Round logRound 1Panel (16 reviewers: Bisky, Hisoka, Mafu-san, Mafuuu, Pariston, Komugi, Gon, Leorio, Chopper, Ging-Go, Knuckle, Meruem, Melody, Kite, Knov, Razor). Netero first pass: 1 P3. Panel: 1 P2, 3 P3, 1 P4, 4 Nit new. 0 dropped. Reviewed against 4968c7e..e26d472. Round 2Churn guard: 7 addressed, 1 contested (CRF-4), 2 acknowledged (CRF-7, CRF-8). Override BLOCKED to PROCEED: author responded to all findings. Panel (5 reviewers: Bisky, Mafuuu, Pariston, Meruem, Razor). CRF-4 closed by panel (4/4). 1 P3 new (CRF-11). Reviewed against 8f4025c..b6365c7. Round 3Churn guard: PROCEED. CRF-11 addressed (7b4f4ee). All findings resolved. Panel (3 reviewers: Bisky, Mafuuu, Pariston). No new findings. Reviewed against 8f4025c..7b4f4ee. About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
There was a problem hiding this comment.
The per-turn reasoning effort feature mirrors the established model_config_id persistence flow cleanly. The SQL batch CTE refactoring from explicit EXISTS to COALESCE + IS DISTINCT FROM is a genuine improvement that avoids dead writes and scales well. Tests cover create, send, validation rejection, and the "absent effort leaves chat unchanged" invariant. The architecture doc was updated in the same commit and matches the implementation.
1 P2, 4 P3, 1 P4, 4 Nit.
The P2 is a gap in appendMessageFields / chatMessage: the new ReasoningEffort field is appended by every other insert path but was missed in the canonical builder used by exported helpers. Currently test-only callers, but the exported contract promises complete params. Five reviewers converged on this independently.
Two design observations from the panel worth noting: (1) once last_reasoning_effort is set, the SQL COALESCE preserves it forever with no reset-to-default path, mirroring the existing last_model_config_id behavior; (2) the API field doc says "for the turn" but the behavior is sticky until the next explicit override. Both match the established pattern and the PR description says this is intentional.
"Real gems, well cut." (Bisky, on the test suite)
This is a first-round panel review.
coderd/x/chatd/chatd.go:3066
P2 [CRF-2] appendMessageFields appends to every InsertChatMessagesParams slice field except the new ReasoningEffort. The chatMessage struct (line 2997) also lacks a reasoningEffort field.
The exported BuildSingleChatMessageInsertParams and BuildSingleUserChatMessageInsertParams produce params where ReasoningEffort is nil while all other slices have one element. PostgreSQL's lockstep UNNEST pads the shorter array with NULLs, so existing test callers pass silently, but this is accidental correctness. The first production caller of these exported builders would silently drop the user's reasoning effort with no compile-time or runtime signal.
"The export surface and the parallel
chatMessagestruct present one face and hide another: they look like the canonical message builder but silently drop a persisted attribute." (Hisoka)
Contrast with recordManualTitleUsage (line 2946), which manually constructs the params and correctly includes ReasoningEffort: []string{""}, showing this was a miss rather than a deliberate omission.
Fix: add a reasoningEffort string field to chatMessage, and append params.ReasoningEffort = append(params.ReasoningEffort, msg.reasoningEffort) in appendMessageFields. The empty-string zero value becomes NULL via the SQL NULLIF(... , ''). (Knuckle P2, Hisoka P3, Mafuuu P3, Meruem P3, Melody P3)
🤖
coderd/database/migrations/000539_chat_reasoning_effort.up.sql:5
P3 [CRF-5] Three columns (chats.last_reasoning_effort, chat_messages.reasoning_effort, chat_queued_messages.reasoning_effort) are plain text with no CHECK constraint. The valid set is seven values. Validation happens only at the HTTP handler layer (IsValidReasoningEffort). Any code path that bypasses the handler (internal callers, migrations, manual fixes) can store arbitrary strings.
Contrast with sibling columns: role uses the chat_message_role enum, plan_mode uses the chat_plan_mode enum. A CHECK (reasoning_effort IS NULL OR reasoning_effort IN ('none','minimal','low','medium','high','xhigh','max')) on each column costs nothing at write time and catches bugs at the source.
The consequence: if validation is bypassed, ResolveReasoningEffort silently falls back to the config default (because reasoningEffortRank won't find the garbage value). The user sees no error; their preference silently disappears. (Knuckle)
🤖
🤖 This review was automatically generated with Coder Agents.
| @@ -3318,13 +3338,23 @@ func (api *API) patchChatMessage(rw http.ResponseWriter, r *http.Request) { | |||
| editModelConfigID = *req.ModelConfigID | |||
There was a problem hiding this comment.
P3 [CRF-1] No test exercises the edit-message reasoning effort path. The validation logic here (lines 3338-3347) and the ReasoningEffort passthrough to EditMessageOptions (line 3354) are untested. The create and send paths both have validation and persistence tests. The edit path's override/preservation behavior in transitions.go:573-576 (use original effort when override is invalid, use override when valid) has zero direct coverage. (Netero)
🤖
There was a problem hiding this comment.
Fixed in 847f0e7.
Generated with Coder Agents.
There was a problem hiding this comment.
Verified fixed in 847f0e72a6. TestPatchChatMessage/ReasoningEffort (with PreservesByDefault and Overrides subtests) and TestEditMessageReasoningEffort now cover the edit-message reasoning effort path at both the HTTP and chatd layers.
🤖
| require.Equal(t, modelConfigB.ID, userMsg.ModelConfigID.UUID) | ||
| } | ||
|
|
||
| func TestSendMessageWithReasoningEffortUpdatesLastReasoningEffort(t *testing.T) { |
There was a problem hiding this comment.
P3 [CRF-3] No test verifies that reasoning effort survives the queued-message roundtrip. TestSendMessageQueuesEffectiveModelConfigID proves model config ID is preserved when a message is queued and later promoted. The PR adds ReasoningEffort to insertQueuedMessage and messageFromQueuedRow (transitions.go:231, 244), but no test queues a message carrying a reasoning effort and verifies the promoted message still carries it.
A bug in either the insert or the promotion would silently drop the user's effort selection. Sketch: clone TestSendMessageQueuesEffectiveModelConfigID, add ReasoningEffort: ptr.Ref("high") to the request, assert the queued message carries it, drain the queue, and assert the promoted chat message does too. (Bisky)
🤖
There was a problem hiding this comment.
Fixed in 847f0e7.
Generated with Coder Agents.
There was a problem hiding this comment.
Verified fixed in 847f0e72a6. TestSendMessageQueuesReasoningEffort verifies the queued row carries the effort and the chat's LastReasoningEffort stays unset. TestPromoteQueuedPreservesReasoningEffort completes the loop: after promotion, both the promoted message and the chat carry the effort.
🤖
| chatReasoningEffort := req.ReasoningEffort | ||
| if chatReasoningEffort != nil && !chatprovider.IsValidReasoningEffort(*chatReasoningEffort) { | ||
| httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{ | ||
| Message: "Invalid reasoning_effort value.", |
There was a problem hiding this comment.
P3 [CRF-4] The three validation sites (lines 1241, 3176, 3344) return "Invalid reasoning_effort value." without including the rejected value. An API consumer debugging a 400 sees their value was wrong but not what value was rejected, forcing correlation with their own request logs. A message like fmt.Sprintf("Invalid reasoning_effort value: %q.", *effort) names the cause.
This follows the existing pattern (plan_mode, busy_behavior do the same), so it's consistent. But the class is wrong: every validation error that rejects user-supplied input should echo it back. (Leorio)
🤖
There was a problem hiding this comment.
Not changed. Treating this as an optional nit for this round.
Generated with Coder Agents.
There was a problem hiding this comment.
Panel closed (4/4 accept). Mafuuu, Pariston, Meruem, and Razor independently verified the convention is consistent across sibling validators (plan_mode, busy_behavior). The consequence (client must check own request body) is acceptable for a small documented enum.
🤖
There was a problem hiding this comment.
I think CRF-4 is valid and we should avoid introducing new bad practices into the code-base.
There was a problem hiding this comment.
Updated the validation response helper so the detail includes the rejected reasoning_effort value.
Generated with Coder Agents.
|
|
||
| rank, ok := reasoningEffortRank(*config.Default) | ||
| effective := requested | ||
| rank, ok := reasoningEffortRank("") |
There was a problem hiding this comment.
Nit [CRF-6] reasoningEffortRank("") always returns (-1, false) since "" is not on the scale. Both rank and ok are unconditionally overwritten before use (either by the effective != nil branch or by the fallback to config.Default). The call is a roundabout variable declaration. Clearer:
effective := requested
var rank int
var ok bool
if effective != nil {
rank, ok = reasoningEffortRank(*effective)
}(Hisoka, Mafuuu, Gon, Chopper, Meruem, Melody, Kite, Knov, Razor, Pariston)
🤖
There was a problem hiding this comment.
Fixed in 847f0e7.
Generated with Coder Agents.
There was a problem hiding this comment.
Verified fixed in 847f0e72a6. ResolveReasoningEffort rewritten with explicit var rank int; var ok bool declarations. Dead initialization eliminated.
🤖
| if chatReasoningEffort != nil && !chatprovider.IsValidReasoningEffort(*chatReasoningEffort) { | ||
| httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{ | ||
| Message: "Invalid reasoning_effort value.", | ||
| Detail: "Must be one of none, minimal, low, medium, high, xhigh, max.", |
There was a problem hiding this comment.
Nit [CRF-7] "Must be one of none, minimal, low, medium, high, xhigh, max." is hardcoded in three places (lines 1242, 3177, 3345). The canonical scale lives in chatprovider.reasoningEffortOrder (unexported). If the scale changes, IsValidReasoningEffort would correctly reject old values but these error messages would list a stale set. Exporting the scale or adding a helper like chatprovider.ReasoningEffortListString() would keep message and validation in sync. (Leorio, Chopper, Razor)
🤖
There was a problem hiding this comment.
Not changed. Treating this as an optional nit for this round.
Generated with Coder Agents.
There was a problem hiding this comment.
If moving to db enums, this becomes a simple join operation. In lieu of that, I think CRF-7 is still valid and should be fixed.
There was a problem hiding this comment.
Replaced the hardcoded allowed-value detail strings with a canonical joined value list from codersdk.ChatModelReasoningEffortValues().
Generated with Coder Agents.
| } | ||
| } | ||
|
|
||
| chatReasoningEffort := req.ReasoningEffort |
There was a problem hiding this comment.
Nit [CRF-8] chatReasoningEffort uses a chat prefix that the sibling variable modelConfigID (line 1122) does not. postChatMessages (line 3173) and patchChatMessage (line 3341) use reasoningEffort and editReasoningEffort matching their modelConfigID / editModelConfigID siblings. Rename to reasoningEffort for consistency. (Gon)
🤖
There was a problem hiding this comment.
Not changed. Treating this as an optional nit for this round.
Generated with Coder Agents.
There was a problem hiding this comment.
I think consistency is worth it tbh.
There was a problem hiding this comment.
Renamed chatReasoningEffort to reasoningEffort for consistency with the sibling modelConfigID naming.
Generated with Coder Agents.
| // Per-turn reasoning effort: the last user-selected effort wins | ||
| // over the model config's default and is clamped to the config's max. | ||
| // Nil when the model config has no reasoning effort configured. |
There was a problem hiding this comment.
Nit [CRF-9] This comment restates ResolveReasoningEffort's own doc comment ("The requested per-turn value wins over the config's default, and the result is clamped to the config's max on the global scale. Returns nil when the model config has no reasoning effort configured"). If the resolution logic changes, two places need updating. The code on lines 538-543 is straightforward: read chat.LastReasoningEffort, pass it to ResolveReasoningEffort. Delete the comment. (Gon)
🤖
There was a problem hiding this comment.
Already resolved.
Generated with Coder Agents.
There was a problem hiding this comment.
Confirmed still resolved. No code change needed for this thread.
Generated with Coder Agents.
| {name: "InvalidDefaultReturnsNil", config: effortConfig(" HIGH ", "high")}, | ||
| {name: "InvalidMaxReturnsNil", config: effortConfig("high", " HIGH ")}, | ||
| {name: "NilConfigIgnoresRequested", requested: ptr.Ref("high")}, | ||
| {name: "DefaultUsedWhenNoRequested", config: effortConfig("medium", "high"), want: ptr.Ref("medium")}, |
There was a problem hiding this comment.
P4 [CRF-10] No test row covers requested with a valid config that has a default but no max. The code path where config.Max == nil and a valid effective value exists (lines 77-85 skipped, line 86 returns effective) is only covered indirectly. A row like {name: "NoMaxReturnsRequested", requested: ptr.Ref("high"), config: effortConfig("low", ""), want: ptr.Ref("high")} would cover the unclamped return. (Bisky)
🤖
There was a problem hiding this comment.
Fixed in 847f0e7.
Generated with Coder Agents.
There was a problem hiding this comment.
Verified fixed in 847f0e72a6. RequestedWinsWithoutMax row added covering the unclamped return path.
🤖
e26d472 to
94e5bb9
Compare
15d4c72 to
228a7db
Compare
94e5bb9 to
8d6506c
Compare
228a7db to
d39150f
Compare
8d6506c to
c4f372b
Compare
d39150f to
355f182
Compare
c4f372b to
28bf556
Compare
355f182 to
3d36549
Compare
28bf556 to
56104c4
Compare
3d36549 to
f677efc
Compare
56104c4 to
51acd5d
Compare
f677efc to
4b77a03
Compare
51acd5d to
474e71e
Compare
4b77a03 to
c485ab1
Compare
474e71e to
514790b
Compare
c485ab1 to
a1bbe9e
Compare
c5b2fa9 to
847f0e7
Compare
|
Review 4639976109: fixed CRF-1/2/3/5/6/10 in 847f0e7. CRF-9 was already resolved. CRF-4/7/8 are optional nits, intentionally not changed. Generated with Coder Agents. |
a1bbe9e to
a1ccef1
Compare
847f0e7 to
515eeef
Compare
a1ccef1 to
8f4025c
Compare
515eeef to
b6365c7
Compare
|
/coder-agents-review |
There was a problem hiding this comment.
Round 2. All R1 findings addressed or closed.
CRF-1/2/3/5/6/9/10: fixes verified in code and tests. CRF-2 (the P2, appendMessageFields gap) is cleanly resolved: the struct carries the field and the function appends it.
CRF-4 (validation error not echoing rejected value): panel closed, 4/4 accept the author's convention defense. The pattern is consistent across plan_mode, busy_behavior, and now reasoning_effort. CRF-7, CRF-8: acknowledged nits, closed.
1 new P3.
"I tried to build a case against this change and couldn't." (Pariston, for the second time)
🤖 This review was automatically generated with Coder Agents.
| {name: "DefaultClampedToMax", config: effortConfig("xhigh", "medium"), want: ptr.Ref("medium")}, | ||
| {name: "InvalidDefaultReturnsNil", config: effortConfig(" HIGH ", "high")}, | ||
| {name: "InvalidMaxReturnsNil", config: effortConfig("high", " HIGH ")}, | ||
| {name: "NilConfigIgnoresRequested", requested: ptr.Ref("high")}, |
There was a problem hiding this comment.
P3 [CRF-11] The old suite had a DefaultClampedToMax row proving that when config.Default exceeds config.Max, the result is clamped to max. That row was removed when the requested parameter was added, and no new row covers the equivalent: requested is nil, default is above max, result should be max.
The existing RequestedClampedToMax tests clamping when the source is requested, and InvalidRequestedFallsBackToDefault tests fallback from invalid requested to default, but neither covers the composition: fallback to default, then clamp to max. The clamping logic operates on the same rank variable regardless of origin, so this is unlikely to break independently, but the guarantee existed before and was dropped.
Fix: add the row back.
{name: "DefaultClampedToMax", config: effortConfig("xhigh", "medium"), want: ptr.Ref("medium")},(Bisky)
🤖
There was a problem hiding this comment.
Restored the missing DefaultClampedToMax row in the amended PR2 commit.
Generated with Coder Agents.
There was a problem hiding this comment.
Verified fixed in 7b4f4eefe6. DefaultClampedToMax row restored: {config: effortConfig("xhigh", "medium"), want: ptr.Ref("medium")} covers the fallback-to-default-then-clamp-to-max path.
🤖
b6365c7 to
7b4f4ee
Compare
|
/coder-agents-review |
There was a problem hiding this comment.
Round 3. All findings resolved.
CRF-11 (the last open finding, missing DefaultClampedToMax test row) fixed in 7b4f4ee. Verified.
Final tally: 11 findings across 3 rounds. 8 fixed, 1 closed by panel (CRF-4, 4/4 accept convention defense), 2 acknowledged nits (CRF-7, CRF-8). No open findings remain.
"I tried to build a case against this and could not." (Pariston)
🤖 This review was automatically generated with Coder Agents.
7b4f4ee to
d34a520
Compare
| CHECK (reasoning_effort IS NULL OR reasoning_effort IN ('none', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max')); | ||
| ALTER TABLE chat_queued_messages | ||
| ADD CONSTRAINT chat_queued_messages_reasoning_effort_check | ||
| CHECK (reasoning_effort IS NULL OR reasoning_effort IN ('none', 'minimal', 'low', 'medium', 'high', 'xhigh', 'max')); |
There was a problem hiding this comment.
Should reasoning_effort just be an enum instead?
There was a problem hiding this comment.
Implemented as a Postgres enum (chat_reasoning_effort) and regenerated the database types/queries.
Generated with Coder Agents.
| }{ | ||
| {name: "NilConfig"}, | ||
| {name: "DefaultUsed", config: effortConfig("medium", "high"), want: ptr.Ref("medium")}, | ||
| {name: "NilConfigIgnoresRequested", requested: ptr.Ref("high")}, |
There was a problem hiding this comment.
We can skip ptr.Ref and just use new("string") instead. Note, db enum as mentioned previously would let us avoid these strings.
There was a problem hiding this comment.
Updated the resolver tests to use new(...) for typed reasoning effort constants, and used new(string(...)) where converting the database enum back to *string is needed.
Generated with Coder Agents.
| return &ptr | ||
| var factoryPtr atomic.Pointer[aibridge.TransportFactory] | ||
| factoryPtr.Store(&factory) | ||
| return &factoryPtr |
There was a problem hiding this comment.
Already resolved, that drive-by rename was not part of this branch after the rebuild.
Generated with Coder Agents.
| } | ||
| } | ||
|
|
||
| chatReasoningEffort := req.ReasoningEffort |
There was a problem hiding this comment.
I think consistency is worth it tbh.
| chatReasoningEffort := req.ReasoningEffort | ||
| if chatReasoningEffort != nil && !chatprovider.IsValidReasoningEffort(*chatReasoningEffort) { | ||
| httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{ | ||
| Message: "Invalid reasoning_effort value.", |
There was a problem hiding this comment.
I think CRF-4 is valid and we should avoid introducing new bad practices into the code-base.
| if chatReasoningEffort != nil && !chatprovider.IsValidReasoningEffort(*chatReasoningEffort) { | ||
| httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{ | ||
| Message: "Invalid reasoning_effort value.", | ||
| Detail: "Must be one of none, minimal, low, medium, high, xhigh, max.", |
There was a problem hiding this comment.
If moving to db enums, this becomes a simple join operation. In lieu of that, I think CRF-7 is still valid and should be fixed.
d34a520 to
b8c5ac8
Compare
| func invalidReasoningEffortResponse(value string) codersdk.Response { | ||
| return codersdk.Response{ | ||
| Message: "Invalid reasoning_effort value.", | ||
| Detail: fmt.Sprintf("got %q, want one of %s", value, allowedReasoningEffortValues), |
There was a problem hiding this comment.
| Detail: fmt.Sprintf("got %q, want one of %s", value, allowedReasoningEffortValues), | |
| Detail: fmt.Sprintf("Invalid value %q, must be one of %s", value, allowedReasoningEffortValues), |
A bit of testing wording bleeding through here. This details is supposed to be helpful to humans hence the suggestion.
There was a problem hiding this comment.
Coder Agents generated.
Updated the wording to Invalid value %q, must be one of %s and updated the tests to match.
b8c5ac8 to
63d94a3
Compare
6038eb2
into
dm/reasoning-effort-1-model-config
Builds on the per-model
reasoning_effort{default, max}config to let users pick the reasoning effort per turn, clamped server-side to the model's max.Changes
reasoning_effortonCreateChatRequest,CreateChatMessageRequest, andEditChatMessageRequest(400 on values outside the global scale);Chatexposeslast_reasoning_effort.model_config_idflow. The value is stored per message (chat_messages.reasoning_effort), survives queueing (chat_queued_messages.reasoning_effort), andInsertChatMessagesupdateschats.last_reasoning_effortfrom the last message in the batch carrying one (unchanged when absent). Editing a message preserves its original effort unless the request overrides it.chats.last_reasoning_effort(else the config default), clamps to the config max, and snaps into the provider's supported set. Models without effort config ignore user-supplied values. Subagent chats intentionally start unset and use the model default.Part of the reasoning effort stack (base: #26974; the chat slider UI follows in #26977).
🤖 Generated by Coder Agents on behalf of @DanielleMaywood