Skip to content

feat(agent,acp): wire provider total_tokens through NIP-AM publish chain - #3593

Merged
wpfleger96 merged 4 commits into
mainfrom
hayt/nipam-provider-totals
Jul 29, 2026
Merged

feat(agent,acp): wire provider total_tokens through NIP-AM publish chain#3593
wpfleger96 merged 4 commits into
mainfrom
hayt/nipam-provider-totals

Conversation

@wpfleger96

@wpfleger96 wpfleger96 commented Jul 29, 2026

Copy link
Copy Markdown
Member

What

Wires genuine provider-reported total_tokens through the full buzz-agent → buzz-acp publish chain so kind-44200 events carry real per-turn and cumulative totals for OpenAI-backed models, while preserving all existing behaviour for Anthropic and external harnesses (goose, claude-code).

Why

Live prod data showed 0 of 1,934 archived reports carry totalTokens. Both hardcoded total_tokens: None in pool.rs and the absent field in buzz-agent's parser are root causes. This is the backend half of a two-track fix; the display-fallback half lands in #2035.

Changes

crates/buzz-agent/src/types.rs

  • Added total_tokens: Option<u64> to LlmResponse with an explicit doc comment that NIP-AM forbids deriving it.
  • Added TurnTotalState enum (Unseen | Exact(u64) | Unknown) with fold() and exact_value() — the tri-state accumulator that distinguishes not-yet-observed from permanently poisoned.

crates/buzz-agent/src/llm.rs

  • parse_responses and parse_openai: read usage.total_tokens from OpenAI Chat Completions (including Databricks routes) and the Responses API via sum_usage.
  • Anthropic: explicit total_tokens: None — no genuine total available; NIP-AM forbids summing categories.

crates/buzz-agent/src/agent.rs

  • Added turn_total_state: &'a mut TurnTotalState to RunCtx.
  • Fold response.total_tokens into the accumulator after each usage-bearing response; non-usage-bearing responses (keepalive/stream frames) do not poison.

crates/buzz-agent/src/lib.rs

  • Added accumulated_total_state: TurnTotalState to Session (default Unseen).
  • Per-turn state passed to RunCtx, folded into session cumulative after each turn.
  • Emits accumulatedTotalTokens in usage_update only when cumulative is Exact(n).

crates/buzz-acp/src/usage.rs

  • Added accumulated_total_tokens: Option<u64> (serde default) to UsageUpdatePayload — optional for goose compat.
  • Added last_total: Option<u64> to SessionState.
  • Added turn_total_tokens and cumulative_total_tokens to TurnUsage (field-local — never affect delta_reliable).
  • Derive turn-total delta only when prev and current are both Some and monotonic; absence, decrease, or no baseline leaves only the total delta null without touching input/output reliability.

crates/buzz-acp/src/pool.rs

  • Replaced both hardcoded total_tokens: None in publish_agent_turn_metric with usage.turn_total_tokens and usage.cumulative_total_tokens.

Tests

20 new tests across the four touched files:

File Tests
types.rs TurnTotalState fold, accumulation, exact_value, default (7 tests)
llm.rs Chat present/absent, Responses present/absent, Anthropic always-None (5 tests)
usage.rs First turn no baseline, second-turn delta, cumulative decrease (field-local), current absent, goose-shaped deserialization, baseline absent (6 tests)
pool.rs Exact turn+cumulative mapping, null totals never derived (2 tests)

cargo test -p buzz-acp -p buzz-agent — all passing, 0 failures.

Scope

Boundary: crates/buzz-agent/** + crates/buzz-acp/** only. Desktop unchanged.
costUsd explicitly out of scope.

Related: #2035

npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw @wpfleger96
feat(agent,acp): wire provider total_tokens through NIP-AM publish chain
56a0a09 
Parse OpenAI-reported total_tokens (Chat Completions + Responses API) in
buzz-agent/llm.rs. Accumulate per-session cumulative total with an explicit
tri-state (Unseen | Exact(n) | Unknown) in buzz-agent/lib.rs: exact only when
every usage-bearing response in the session supplied a provider total; unknown
after the first missing total, until a new session resets. Anthropic stays None
(no genuine total; NIP-AM forbids summing categories).

Extend buzz-acp/usage.rs UsageUpdatePayload with an optional
accumulatedTotalTokens field (goose compat preserved). Track previous/current
snapshots in SessionState; derive a turn delta only when both snapshots are
present and monotonic -- absence, decrease, or no baseline leaves only the
total delta null. Missing totals never flip deltaReliable or discard reliable
input/output/cache deltas. Map turn and cumulative totals into the two
TokenCounts fields in buzz-acp/pool.rs (replacing hardcoded None).

Required tests added: Chat/Responses total_tokens parsing; multiple provider
rounds all-exact; mixed present/missing totals within one turn; permanent
session poisoning + new-session reset; first update without a baseline;
cumulative decrease; omitted optional field from goose-shaped JSON; exact
turn/cumulative publisher JSON mapping. 20 new tests, all passing.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96
wpfleger96 requested a review from a team as a code owner July 29, 2026 18:30
npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw and others added 3 commits July 29, 2026 16:43
npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw @wpfleger96
chore(agent,acp): fix fmt and clippy on provider totalTokens tests
7504216 
Run cargo fmt --all to resolve 5 style hunks in pool.rs, usage.rs,
lib.rs, and types.rs. Drop the redundant Some()/unwrap() wrapping in
the null-totals-never-derived test (clippy::unwrap_used on a
known-Some value); bind TokenCounts directly instead.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw @wpfleger96
fix(agent,acp): checked-add overflow poison, session-boundary tests, …
58aca40 
…honest publisher tests

Overflow: replace saturating_add with checked_add in TurnTotalState::fold()
and the turn→session merge. Overflow now produces Unknown rather than
Exact(u64::MAX), preserving the exact-only invariant for both boundaries.

Centralize: extract TurnTotalState::merge_session() so the per-response fold
and the turn→session boundary share one checked-add implementation. Replace
the open-coded match in lib.rs with a call to merge_session().

Session-boundary tests: add two integration tests in fake_llm.rs that drive
the Session.accumulated_total_state match through the wire harness:
- session_total_poisoned_by_missing_total_and_stays_poisoned: verifies
  Exact→Unknown transition and permanent poisoning, with i/o unaffected.
- new_session_resets_total_accumulation: verifies accumulated_total_state
  is Unseen on session/new, not inherited from a prior session.

Publisher tests: extract build_turn_metric_counts() as a pub(crate) pure
function called by publish_agent_turn_metric(). Replace the two tests that
duplicated the production logic with tests that call the real builder and
assert on serialized JSON field names (camelCase) and values. Reverting
the production builder fields to None now breaks these tests.

Minor: add an unexpected total_tokens field to the Anthropic parse test
fixture so the test would catch future accidental parsing. Add a shape-
assumption comment to agent.rs explaining the input-or-output usage gate.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
npub1g8493u0xfsjrvflg4n08ezd7vec99mnwzlv0qgwpr9d7gvjwhuzqx59rhw @wpfleger96
refactor(agent): extract checked_exact_sum to centralize overflow sem…
dbc5500 
…antics

The acc.checked_add(n) → Exact|Unknown match was duplicated in fold() and
merge_session(). Extract a private checked_exact_sum(acc, n) helper and
have both arms delegate to it. Overflow semantics now have exactly one
definition; a future change needs to be made in one place only.

Update comments in fold() and merge_session() to reference the helper
rather than claiming each independently implements the contract.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96
wpfleger96 merged commit f95fdc1 into main Jul 29, 2026
32 checks passed
@wpfleger96
wpfleger96 deleted the hayt/nipam-provider-totals branch July 29, 2026 22:15
@wesbillman wesbillman mentioned this pull request Jul 29, 2026
Merged
joahg added a commit to joahg/buzz-dev-mode that referenced this pull request Jul 29, 2026
@joahg
Merge remote-tracking branch 'origin/main' into joah/dev-mode-display…
9fe5252 
…-style

* origin/main:
  feat: add first-class OpenRouter provider support (block#1975)
  feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (block#3593)

Co-authored-by: Amp <amp@ampcode.com>
Signed-off-by: Joah Gerstenberg <joah@squareup.com>
Amp-Thread-ID: https://ampcode.com/threads/T-019fabca-7cdc-777d-9ccd-ed829c1e60f0

# Conflicts:
#	desktop/src-tauri/src/commands/agent_models.rs
tlongwell-block pushed a commit that referenced this pull request Jul 30, 2026
npub1qyvc0c5kl4gqv2fd97fsk46tu378sqgy35vc83rvgfwne90sel7s0ed67d Tyler Longwell
Merge remote-tracking branch 'origin/main' into eva/agent-trading-cards
b176a5d 
* origin/main: (29 commits)
  feat(replica): portable heartbeat-token fence with snapshot-local reader routing (#3268)
  fix(git): channel binding tooling + author remediation for unbound repos (#3626)
  feat: configure S3 URL addressing style (#3400)
  feat: add first-class OpenRouter provider support (#1975)
  feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (#3593)
  chore(release): release Buzz Desktop version 0.5.2 (#3624)
  docs: add Linux rendering troubleshooting guide (#3573)
  fix(desktop): discover bun-installed agent CLIs in ~/.bun/bin (#3343)
  feat(tracing): correlate trace IDs in relay logs (#3608)
  chore(ci): bump Linux AppImage build container to ubuntu:24.04 (#3602)
  feat(cli): mirror Desktop mention delivery (#3330)
  fix(desktop): deduplicate relay outage notification (#3579)
  fix(desktop): reconcile thread arrivals at bottom (#3585)
  fix(mobile): keep TLS on relays joined by invite (#3139)
  Improve emoji autocomplete matching (#3571)
  Fix shared agent avatar import profiles (#3578)
  Fix inline raster avatars in agent catalog (#3581)
  revert(acp): remove dead GOOSE_ACP_SCHEDULER_DISABLED env injection (#3576)
  feat(agent): make Gemini and MLflow-route models usable through databricks_v2 (#3569)
  fix(cli): mask credential env values in --help output (#3570)
  ...

Co-authored-by: Tyler Longwell <tlongwell@block.xyz>
Signed-off-by: Tyler Longwell <tlongwell@block.xyz>
wpfleger96 added a commit that referenced this pull request Jul 30, 2026
@wpfleger96
Merge remote-tracking branch 'origin/main' into duncan/agent-usage-ar…
55dabfd 
…chive

* origin/main: (22 commits)
  feat(catalog): resolve publisher display name in catalog detail pane (#3640)
  feat(mesh): upgrade embedded mesh to v0.74 and harden shared compute (split 1/2 of #3467) (#3741)
  docs(nips): specify kind:30621 multi-repo projects (NIP-MP) (#3163)
  Refine agent sharing dialog (#3699)
  desktop: enable getUserMedia in the Linux WebKitGTK webview (#3607)
  fix: align responsive agent views (#3688)
  Add macOS agent menu-bar menu (#3565)
  Fix pending message feedback (#3543)
  fix(desktop): remove remaining Projects panel fills (#3742)
  feat(mobile): desktop-parity emoji and thread experience (#3485)
  desktop: restore direct community member adds (#3634)
  fix(desktop): explain open agent access (#2561)
  fix(cli): resolve agents from owner records (#3178)
  fix(desktop): remove Projects overview card fills (#3416)
  feat(replica): portable heartbeat-token fence with snapshot-local reader routing (#3268)
  fix(git): channel binding tooling + author remediation for unbound repos (#3626)
  feat: configure S3 URL addressing style (#3400)
  feat: add first-class OpenRouter provider support (#1975)
  feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (#3593)
  chore(release): release Buzz Desktop version 0.5.2 (#3624)
  ...

Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
wpfleger96 pushed a commit that referenced this pull request Jul 30, 2026
npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7
Merge remote-tracking branch 'origin/main' into duncan/install-ceilin…
93ccd19 
…g-pipeline

* origin/main: (25 commits)
  Refine agent sharing dialog (#3699)
  desktop: enable getUserMedia in the Linux WebKitGTK webview (#3607)
  fix: align responsive agent views (#3688)
  Add macOS agent menu-bar menu (#3565)
  Fix pending message feedback (#3543)
  fix(desktop): remove remaining Projects panel fills (#3742)
  feat(mobile): desktop-parity emoji and thread experience (#3485)
  desktop: restore direct community member adds (#3634)
  fix(desktop): explain open agent access (#2561)
  fix(cli): resolve agents from owner records (#3178)
  fix(desktop): remove Projects overview card fills (#3416)
  feat(replica): portable heartbeat-token fence with snapshot-local reader routing (#3268)
  fix(git): channel binding tooling + author remediation for unbound repos (#3626)
  feat: configure S3 URL addressing style (#3400)
  feat: add first-class OpenRouter provider support (#1975)
  feat(agent,acp): wire provider total_tokens through NIP-AM publish chain (#3593)
  chore(release): release Buzz Desktop version 0.5.2 (#3624)
  docs: add Linux rendering troubleshooting guide (#3573)
  fix(desktop): discover bun-installed agent CLIs in ~/.bun/bin (#3343)
  feat(tracing): correlate trace IDs in relay logs (#3608)
  ...

Signed-off-by: npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@wpfleger96