feat: allow admins to override the chat compaction model - #27151
Conversation
Adds a deployment-wide 'compaction' context to the chat model override framework. When set, chatd generates compaction summaries with the override model instead of the chat model. Thresholds use the stricter of the two context limits, the prompt is re-sanitized for the override provider, and metrics plus debug runs record the model that actually generated the summary.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 89f78c7369
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Resolving the compaction model override previously built the route and model client on every generation prepare, so a configured but temporarily broken override could fail ordinary turns that never compact. Split the resolver: the stored config (whose context limit feeds the compaction trigger) is resolved on every prepare with soft fallback, while route and client construction happen only once compaction is known to run.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c488f6b6f5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Adding the compaction context to AllChatModelOverrideContexts extends the invalid-context error detail, so the UnknownContextReturns400 expectations must list it.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 358739808c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
compactionNeeded at prepare time does not guarantee compaction runs: decideGenerationAction can return FinishTurn first (completed step, stop-after tool, max steps), and building the override client during prepare failed such turns on hard construction errors. Carry the resolved override config through generationCompaction and build the client, swap identity, and sanitize the prompt inside generateCompaction, which only runs for the compact action.
|
@codex review |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
# Conflicts: # coderd/x/chatd/ARCHITECTURE.md
|
@codex review |
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…ction Align the cross-provider compaction prompt pass with coder/mux's flattenProviderExecutedToolParts: rewrite provider-executed tool calls and results into plain text parts instead of dropping them, so the summary keeps server-side tool content (for example web_search results) while still shedding the provider-specific wire shape other providers reject. Anomalous provider-executed parts outside assistant messages are still dropped.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f9bbc7b181
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The compaction override resolver folds the admin-configured reasoning effort into the override config's call options, but the summary call never consumed them. Convert the override's provider options and resolved reasoning effort when building the override client and pass them on the summary fantasy.Call, so a compaction model's configured effort actually applies.
|
@codex review |
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…anitize.go The file never defined the compaction prompt; the summary prompt text lives in chatloop/compaction.go. It sanitizes the already-built prompt messages for the compaction override model, so name it after its entry point sanitizeCompactionPrompt and the chatsanitize package it wraps.
…the override model The still-over-limit terminal error is recorded at decision time, before the compact action builds the override client, so it used the chat model labels from the prepared options while the compact action's own events used the built override client's labels. A failing override compaction then split across two metric series: success on the override model, error on the chat model. Prepare-time override resolution already computes the normalized provider and validates ResolveModelWithProviderHint, so keep that identity in resolvedCompactionOverride and source every compaction metric label from one compactionMetricIdentity helper.
|
@codex review |
Documentation CheckUpdates Needed
Automated review via Coder Agents |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e4a684d89b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The compaction trigger uses the stricter of the chat and override models' context limits, but the still-over-limit check compared against the chat model's limit. A stricter override then looped through repeated compactions instead of failing the turn.
|
@codex review |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Chat compaction currently summarizes the conversation with the same model that runs the chat turn, and admins have no way to route compaction to a stronger model when the chat model produces poor summaries.
This adds a deployment-level compaction model override, configured on the AI Settings > Coder Agents page next to the existing title generation override (site config key
agents_chat_compaction_model_override, served by the existing/api/experimental/chats/config/model-override/{context}endpoints). When set, compaction summaries are generated with the override model while the chat keeps its own model for the continuation turn.Behavior details:
Closes CODAGT-339