fix(buzz-agent): Responses reasoning summary, Anthropic display:summarized, ACP v2 messageId - #5195
Conversation
… reasoning summaries
The OpenAI Responses API only populates reasoning summary arrays when a
summary mode is explicitly requested via `reasoning.summary`. Without it,
GPT-family models through buzz-agent bill thinking tokens but return
`summary: []`, leaving the observer feed empty even with effort set.
Changes:
- config.rs: new `ThinkingSummary` enum (Auto/Concise/Detailed) and
`parse_thinking_summary` fn mirroring `parse_thinking_effort`; new
`thinking_summary` field on `Config` (default Auto); wired via
BUZZ_AGENT_THINKING_SUMMARY env var.
- llm.rs: `responses_body` now emits `reasoning.summary` alongside
`reasoning.effort` when effort is set. No summary sent when effort is
None — avoids 400s on non-reasoning models.
- env_vars.rs: BUZZ_AGENT_THINKING_SUMMARY added to is_safe_to_reveal
allowlist (non-secret enum, same treatment as THINKING_EFFORT).
- agent_config_tests.rs: extended allowlist tests with THINKING_SUMMARY.
Also resolves the doc contradiction in is_adaptive_thinking_model vs
anthropic_thinking_config: the roster comment previously implied adaptive
models reason without any config ("always-on"), conflicting with the
anthropic_thinking_config doc saying `thinking:{type:"adaptive"}` is
required. Clarified: some models reason by default (Fable 5, Mythos 5,
Mythos Preview per the extended-thinking support table), but we always
send type:"adaptive" explicitly so that output_config.effort is honoured
and thinking token depth is predictable. Source:
https://platform.claude.com/docs/en/build-with-claude/extended-thinking
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…accuracy Three pre-existing gaps resolved per Thufir review findings on PR #5195. Fix A — config.rs comment accuracy (Thufir finding 1a): Rewrite the anthropic_thinking_config doc and is_adaptive_thinking_model roster to match the per-model table at: https://platform.claude.com/docs/en/build-with-claude/thinking-troubleshooting#supported-models Correct split: - Opus 4.6/4.7/4.8, Sonnet 4.6: thinking OFF by default; type:adaptive required. - Opus 5, Sonnet 5, Fable 5, Mythos 5, Mythos Preview: thinking ON with no config (always-on); we still send type:adaptive to activate output_config.effort. Previous comments claimed all adaptive families needed the field to think — true only for the first sub-bucket. Fix B — Anthropic thinking display (Thufir finding 1b / root cause of observer gap): Anthropic defaults thinking.display to omitted on newest models (Fable 5, Mythos 5, Opus 5, Sonnet 5, Opus 4.8, Opus 4.7, Mythos Preview), returning thinking blocks with an empty thinking field — parse_anthropic read nothing. Fix: send display:summarized in anthropic_thinking_config() for both the adaptive shape and the manual-budget shape whenever thinking is enabled. Tests: adaptive and manual-budget families each assert display:summarized present; unknown model path asserts thinking absent. DBv2 gateway parity for display unverified — flagged in PR body. Fix C — ACP v2 ContentChunk messageId compliance (Thufir ACP addendum): buzz-agent negotiates ACP v2 but agent.rs emitted agent_thought_chunk and agent_message_chunk without messageId. ACP v2 ContentChunk requires both messageId and content (schema/v2/schema.json @d13d1baa); v1 allows the field, so this addition is backwards-safe and buzz-acp/Desktop already parse it. Fix: assign a stable round-scoped ID (format round-{n}) shared by the thought and assistant message chunks from the same provider round. A provider round produces at most one of each, so one ID per round is the right granularity. Integration test: negotiates v2, drives agent through a Responses-route reasoning response, asserts both chunk types carry identical non-empty messageId plus correct content text. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ys-on terminology, lint clean Three targeted repairs per Thufir round-2 review of PR #5195. Fix 1 — ACP v2 messageId contract (Thufir IMPORTANT, corrects round-2 implementation): The prior round-2 gave both the thought and assistant chunks from one provider round the same messageId ('round-{n}'), which is wrong in two ways: (a) they are two distinct logical messages per the ACP v2 Message ID RFD; (b) 'round' resets per run() so 'round-1' recurred across session/prompt calls in the same ACP session, violating the session-uniqueness requirement. Fix: derive two IDs per round from the existing per-run run_id (already random per session/prompt, lib.rs:842): '<run_id>-thought-<round>' and '<run_id>-message-<round>'. Added run_id to RunCtx so agent.rs can access it without a new dependency. Plumbed from lib.rs run_prompt (run_id was already in scope, just not threaded through RunCtx). Updated test_acp_v2_chunks_carry_message_id to assert the corrected invariants: thought and message IDs are distinct, both are non-empty, and neither recurs across a second session/prompt in the same ACP session (the cross-prompt case the old test did not exercise). Fix 2 — Three-way On/Always-on/Off terminology (Thufir IMPORTANT, completes Fix A): Anthropic's support table uses three distinct statuses, not two. The round-2 pass collapsed all non-Off models into 'always-on', incorrectly calling Opus 5 and Sonnet 5 'always-on' (their status is 'On': default-on, can be disabled) and calling Mythos Preview 'default-on' (its status is 'Always on': cannot be disabled). Also, the inline branch comment at config.rs:167 still said 'thinking must be explicitly enabled', contradicting On/Always-on models. Corrected all four locations: config.rs top-of-function doc (sub-bucket list), is_adaptive_thinking_model doc (sub-bucket list), inline branch comment, and inline code comments at the Fable 5/Mythos 5/Mythos Preview match arms. Fix 3 — doc_lazy_continuation lint (Thufir IMPORTANT, fixes failing CI gate): Three clippy::doc_lazy_continuation errors at the continuation lines after the nested list in the anthropic_thinking_config doc comment. Added a blank doc line after the list items to terminate the list context before the continuation prose. Source: Anthropic thinking-troubleshooting#supported-models table (On/Always-on distinction); ACP Message ID RFD at d13d1baa lines 63-67, 115-124, 256-263. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wesbillman
left a comment
There was a problem hiding this comment.
Reviewed on Wes's behalf at exact head 47ef696d6094d5fa4adcdc5a0a3dbe63de0a2727. No blocking findings.
I traced all three changes through request construction, provider-response parsing, and ACP emission:
- OpenAI Responses:
reasoning.summaryis emitted only alongside configuredreasoning.effort;parse_responsesalready consumes everysummary_textitem. Defaultautois the provider-recommended way to select the most detailed summary supported by the model. Explicitconcise/detailedremain operator choices because OpenAI documents model-dependent support. - Anthropic:
display: "summarized"is valid in bothtype: "adaptive"andtype: "enabled"shapes. Anthropic's current thinking documentation explicitly supports both modes and explains that newer-model defaults otherwise return an emptythinkingfield. The field remains absent when this code does not enable/configure thinking. - ACP v2: each emitted thought and assistant message gets a distinct ID, stable across its chunks and unique across rounds and prompt invocations because the prefix is the existing fresh per-prompt
run_id. The two-prompt golden test pins the collision boundary that the earlier implementation missed.
The new env value is non-secret and its reveal allowlist/config parsing are covered. git diff --check passed. Full relevant CI is green (Rust Lint, Unit Tests, Desktop Core/build/smoke, Mobile, Windows Rust, cross-compiles, Security); two Desktop integration shards were still running when reviewed. The author-disclosed DBv2 Anthropic gateway passthrough uncertainty may affect whether that route gains observer text, but it does not create a regression in the native Anthropic path and is not a merge blocker.
Verdict: 9/10, safe to merge once required CI completes. This is a comment-only review, not an approval.
generate run_id before s.busy = true in acquire_session(). if session_token() fails, propagate a static error and reject the prompt without mutating session state. previously unwrap_or_else fell back to "x" making every prompt in the session share the same run_id namespace, recreating the within-session messageId collisions this PR eliminates. three comment fixes ride along: - config.rs:114: "both sub-buckets" -> "all three sub-buckets" - config.rs:2333: "default-on" -> "Always on" for Mythos Preview test - golden_transcripts.rs:822: remove overclaiming stability assertion (test collects one chunk per kind per prompt, not repeated chunks) Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…-log-harness * origin/main: feat(desktop): adding rich link previews to messages (#3818) fix(buzz-agent): Responses reasoning summary, Anthropic display:summarized, ACP v2 messageId (#5195) fix(desktop): retain distinct agent instances in autocomplete (#5202) fix(desktop): defer channel visibility change to Save (#5203) feat(desktop): Projects follow-ups — access restrictions, fast loading, activity feed polish (#5073) refactor(cli): replace probe/decider/detail split with single typed extractor (#5191) fix(desktop): drop unhandled rejection from throwing window.Notification (#5143) fix(desktop): fence localStorage SecurityError from killing the React tree (#5142) fix(desktop): make terminal output selectable (#4980) fix(desktop): use WEBKIT_DMABUF_RENDERER_FORCE_SHM for NVIDIA/AppImage (#3654) (#4505) Make public starter channels best effort (#5192) Mobile: add anchored reaction popover (#5025) feat(mobile): add bee pull-to-refresh (#5059) Signed-off-by: Atish Patel <atish@squareup.com>
* origin/main: (32 commits) Recover from max-token response truncation (#5223) chore(release): release Buzz Desktop version 0.5.6 (#5214) fix(mobile): keep latest messages above composer (#4981) fix(sdk): preserve self-mention p tags in message and forum event builders (#4975) bump @tauri-apps/cli to ~2.11.4 to fix linux app icon issue (#4858) feat(desktop): adding rich link previews to messages (#3818) fix(buzz-agent): Responses reasoning summary, Anthropic display:summarized, ACP v2 messageId (#5195) fix(desktop): retain distinct agent instances in autocomplete (#5202) fix(desktop): defer channel visibility change to Save (#5203) feat(desktop): Projects follow-ups — access restrictions, fast loading, activity feed polish (#5073) refactor(cli): replace probe/decider/detail split with single typed extractor (#5191) fix(desktop): drop unhandled rejection from throwing window.Notification (#5143) fix(desktop): fence localStorage SecurityError from killing the React tree (#5142) fix(desktop): make terminal output selectable (#4980) fix(desktop): use WEBKIT_DMABUF_RENDERER_FORCE_SHM for NVIDIA/AppImage (#3654) (#4505) Make public starter channels best effort (#5192) Mobile: add anchored reaction popover (#5025) feat(mobile): add bee pull-to-refresh (#5059) Remove agent creation success modal (#5063) fix(buzz-agent): escalate LLM timeouts per retry and log per-call latency (#5130) ... Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com> # Conflicts: # desktop/src/shared/api/tauri.ts
Brings the bench branch up from 13c9e90 to current main (c7b6636) as a merge commit — no rebase, no history rewrite. Clean auto-merge, zero conflicts. Notably picks up: - #5248: budget summarizer reasoning separately so it cannot starve the handoff summary - #5223: recover from max-token response truncation - #5195: Responses reasoning summary + ACP v2 messageId fixes - #5130: escalate LLM timeouts per retry, log per-call latency Originating Buzz thread: buzz://message?channel=c3252dd2-0142-4e01-88c7-a2183c3960a5&id=9e60a8dfa64a59339d7b357c509e231f6bcf76e5507a92078a6723378f9d1e95 Co-authored-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz> Signed-off-by: Eva <011987e296fd5006292d2f930b574be47c7801048d1983c46c425d3c95f0cffd@buzz.block.builderlab.xyz>
…format * origin/main: (60 commits) feat(desktop): unify add agent flows (#5015) fix(buzz-agent): budget summarizer reasoning separately so it cannot starve the handoff summary (#5248) infra: bind development services to loopback (#4871) chore(release): release Buzz Desktop version 0.5.7 (#5252) fix(desktop): isolate relay admission tests (#5221) fix(desktop): externalize boot <style> to prevent Tauri CSP nonce override (#5242) fix(desktop): let imported and recovered identities finish onboarding (#5228) Recover from max-token response truncation (#5223) chore(release): release Buzz Desktop version 0.5.6 (#5214) fix(mobile): keep latest messages above composer (#4981) fix(sdk): preserve self-mention p tags in message and forum event builders (#4975) bump @tauri-apps/cli to ~2.11.4 to fix linux app icon issue (#4858) feat(desktop): adding rich link previews to messages (#3818) fix(buzz-agent): Responses reasoning summary, Anthropic display:summarized, ACP v2 messageId (#5195) fix(desktop): retain distinct agent instances in autocomplete (#5202) fix(desktop): defer channel visibility change to Save (#5203) feat(desktop): Projects follow-ups — access restrictions, fast loading, activity feed polish (#5073) refactor(cli): replace probe/decider/detail split with single typed extractor (#5191) fix(desktop): drop unhandled rejection from throwing window.Notification (#5143) fix(desktop): fence localStorage SecurityError from killing the React tree (#5142) ... Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Three pre-existing gaps in the buzz-agent observer feed fixed together per Will's ruling ("all 3 in the current PR"):
responses_bodynever requestedreasoning.summary; GPT-family models billed thinking tokens but returnedsummary: [].anthropic_thinking_config()never sentthinking.display; newest Claude models (Opus 5, Sonnet 5, Fable 5, Mythos 5, Opus 4.7/4.8, Mythos Preview) default todisplay:"omitted", returning thinking blocks with an emptythinkingfield — observer rendered nothing.agent_thought_chunkandagent_message_chunkwithoutmessageId, which ACP v2'sContentChunkrequires (messageId+contentboth required at schema headd13d1baa).Changes
crates/buzz-agent/src/config.rsThinkingSummaryenum (Auto/Concise/Detailed) withBUZZ_AGENT_THINKING_SUMMARYenv var (defaultAuto); mirrorsBUZZ_AGENT_THINKING_EFFORTpatternanthropic_thinking_config()now emits"display": "summarized"in both the adaptive shape and the manual-budget shape whenever thinking is enabledis_adaptive_thinking_modelandanthropic_thinking_configdoc comments to match Anthropic's exact three-way per-model terminology (doc: https://platform.claude.com/docs/en/build-with-claude/thinking-troubleshooting#supported-models):type:"adaptive"required to enabletype:"adaptive"to activateoutput_config.efforttype:"adaptive"to activateoutput_config.effortcrates/buzz-agent/src/llm.rsresponses_bodyemitsreasoning.summaryalongsidereasoning.effortwhen effort is set (gated — no barereasoning:{summary}without effort)crates/buzz-agent/src/agent.rsagent_thought_chunkcarries"messageId": format!("{run_id}-thought-{round}")agent_message_chunkcarries"messageId": format!("{run_id}-message-{round}")run_idis a fresh random token persession/promptinvocation so IDs are session-unique across multiple promptscrates/buzz-agent/src/lib.rsrun_idplumbed intoRunCtx(was already generated inrun_prompt, just not threaded through)crates/buzz-agent/tests/golden_transcripts.rstest_acp_v2_chunks_carry_message_id— negotiates v2, drives two consecutivesession/promptcalls, asserts: both chunk types carry non-emptymessageId; thought and message IDs are distinct; IDs do not recur across the two prompts in the same ACP sessiondesktop/src-tauri/src/managed_agents/env_vars.rsBUZZ_AGENT_THINKING_SUMMARYadded tois_safe_to_revealallowlistdesktop/src-tauri/src/commands/agent_config_tests.rsBUZZ_AGENT_THINKING_SUMMARYallowlist entry (case-insensitive)Tests added
parse_thinking_summary_round_trips_all_valuesparse_thinking_summary_unset_and_empty_yield_autoparse_thinking_summary_is_case_insensitiveparse_thinking_summary_rejects_unknown_valuethinking_summary_as_str_mappingresponses_body_summary_present_iff_effort_setresponses_body_emits_configured_summary_moderesponses_body_concise_summary_modeanthropic_thinking_config_adaptive_emits_display_summarizedanthropic_thinking_config_manual_budget_emits_display_summarizedtest_acp_v2_chunks_carry_message_id(integration test — two-prompt cross-session case)Notes
display: unverified — the DBv2 Claude route proxies Anthropic Messages shape, but whether the gateway passesthinking.displaythrough is not confirmed. Flagged here rather than blocking on it.messageIdas optional and will pick it up from the wire automatically.