Skip to content

test(budget): pin the usage-attribution invariant — child cost bills, child context never leaks (#418) - #465

Merged
justrach merged 2 commits into
release/0.0.242from
fix/418-usage-attribution
Aug 6, 2026
Merged

test(budget): pin the usage-attribution invariant — child cost bills, child context never leaks (#418)#465
justrach merged 2 commits into
release/0.0.242from
fix/418-usage-attribution

Conversation

@justrach

@justrach justrach commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Closes #418. Audit outcome: no bug. The invariant already holds, and it holds structurally rather than by a discipline someone could forget. The test is the deliverable.

The question, answered

Does a child's usage ever reach the parent's context estimate, as opposed to only its billing totals?

No. The two accounts are separate by construction:

  • Bill is process-wide. Agent.recordCost writes pricing.g_cost, a global tally. A child bills the run automatically — that is why the [usage] footer includes the fleet.
  • Meter is per-Agent. last_context_tokens / context_local_tokens are fields on the Agent that made the request. runSub gives the child a fresh Agent with both at their 0 defaults, and ToolCtx carries no pointer back to the parent Agent — so a child has nothing to write to even if it tried.
  • Compaction inputs are parent-local. inputOverCompactThresholdeffectiveContextTokenscontextEstimate reads only self.messages, self.systemPrompt(), self.toolsJson() and the two self anchors, and both gate call sites pass root.
  • run_budget.zig / phase_budget.zig have no token field at all — they ledger model calls, depth and concurrency. There is nothing there to conflate.

All 7 runSub call sites were checked, across both spawn paths: foreground execSubagent keeps run.output and drops run.usage; background agentJobPump stores job.usage and agent_output renders it as text; the workflow/retry/judge/pipeline sites all return run.output. No asymmetry between them, which is where a bug of this shape would most likely hide.

Worth noting the layers that do reuse the root Agent already implement exactly the capture-and-pin-back the issue describes: review.Context.begin/restore and compact().

The tests

parent context is identical with and without an attributed child; only the bill differs — two identically seeded roots, one of which runs eight workers each ending at 164k context, deliberately past the parent's 80k compactAt() so a single leaked token would trip the gate. Asserts last_context_tokens, context_local_tokens, effectiveContextTokens() and inputOverCompactThreshold() all equal the control field-for-field, and that g_cost moved by exactly 8× the children's in/cache/out tokens and USD.

a child's context total reaches the parent as report bytes, never as context — a child reporting 480k has its status line appended as a tool result; the parent grows by under 200 tokens rather than 480k, and stays under threshold.

One latent trap found, documented rather than changed

behavior_upload.zig sums context_tokens across parent and subagent API calls, so a fleet turn's TurnMetrics.context_tokens is a sum-of-agents, not an occupancy. It is write-only telemetry that no harness decision reads back, so it is not a bug today — but it is exactly the field someone would later mistake for a context reading, so it now says so in place.

Verification

zig build test1118/1118, +2 from the 1116 baseline. scripts/eval-tier1.sh green (reachability confirms 1079 declared tests compiled in).

Not traced: whether the GUI/--json frontend aggregates the agent_usage event's context_tokens for display. That is a wire event to a separate process and cannot feed the Zig meter, so it is outside this invariant, but the GUI side was not read.

justrach and others added 2 commits August 6, 2026 20:57
#418 asked whether graff ever lets a subagent's tokens inflate its
parent's context-window estimate, which would fire compaction early on
fleet-heavy runs. Audited the four named surfaces and the answer is no,
by construction rather than by a save/restore dance:

  * the BILL is process-wide. Agent.recordCost writes pricing.g_cost, so
    a child's spend lands in the [usage] footer, /cost and the --json
    cost_usd without anyone forwarding it.
  * the METER is per-Agent. last_context_tokens / context_local_tokens
    are fields on the Agent that made the request; runSub builds the
    child its OWN Agent, and ToolCtx carries no pointer back to the
    parent, so a child has nothing to write to.
  * run_budget/phase_budget ledger model CALLS, not tokens. Neither has
    a token field to conflate.

A completed child hands back SubRun{output, usage}. The foreground path
(execSubagent) keeps output alone; the background path (agentJobPump ->
agent_output) stores usage on the job and renders it through
agentStatusText -- as text the model reads, never as a number the
parent's meter adds. The in-place layers that DO reuse the root Agent
(review.Context, compact()) already capture and pin the meter back.

Nothing to fix, so the test is the deliverable: two parents seeded
identically, one of them fleeted with eight workers that each end past
the parent's compaction threshold. Their meters stay equal field for
field, neither trips inputOverCompactThreshold, and the run tally moves
by exactly the children's tokens and USD. The second test pins the one
place a child's context total does cross over -- the report line -- to
the bytes it actually costs, not the 480k it names.

Suite: 1118 pass (was 1116). Tier 1 green.

Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
Found while auditing #418. behavior_upload.recordApi accumulates the
CALLER's meter into one turn-wide total, root and subagent alike, so a
turn that fanned out eight workers reports roughly nine windows' worth
under a field named context_tokens. Correct for upload, and never read
back by the harness -- but it is exactly the field a future reader
reaches for when they want "how full was the context on this turn", and
the name gives them no warning. Answering that from here would
re-introduce the bug #418 exists to prevent, one layer over.

Documented at the type (every field is a sum over the turn's agents), at
the field (the authoritative per-agent occupancy is
Agent.last_context_tokens / effectiveContextTokens, which is what the
compaction gates read), and at the accumulator (the summing is deliberate
and belongs on this side of the boundary).

Comments only. Suite still 1118 pass; tier 1 green.

Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
@justrach
justrach merged commit cd3caed into release/0.0.242 Aug 6, 2026
5 of 6 checks passed
@justrach
justrach deleted the fix/418-usage-attribution branch August 6, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant