Skip to content

feat(compact): reserve a buffer and write notes-to-self before context rollover (#391) - #460

Merged
justrach merged 2 commits into
release/0.0.242from
feat/391-pre-compaction-notes
Aug 6, 2026
Merged

feat(compact): reserve a buffer and write notes-to-self before context rollover (#391)#460
justrach merged 2 commits into
release/0.0.242from
feat/391-pre-compaction-notes

Conversation

@justrach

@justrach justrach commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Closes #391. The Codex auto_compact_fallback_buffer_tokens / auto_compact_fallback_prompt pattern: the agent writes what its future self needs before compaction, instead of losing working state silently.

The reserve shares #390's ledger, it does not add a second one

This was the issue's hardest constraint, so it gets the detail. The reservation lives in src/phase_budget.zig, the ledger over the shared RunBudget pool where #390's landing reserve already lives (landingReserve, Ledger.reserve; its hard half is the depth > 0 and remaining() <= 1 guard in run_budget.zig:97). The addition is cost_precompact_note: u64 = 1 plus Ledger.affordsHarnessNote(remaining), which is literally fits(remaining, cost_precompact_note) — the same "must fit on top of the reserve" predicate subagent.scoreVariants uses for judges.

The note is deliberately the junior liability: narration is mandatory and owns the reserve, the note is optional and must clear it. A separate reserve would have been a second ledger over one pool, double-counting the last call.

Confirmed three ways rather than asserted: decideCalls constructs a phase_budget.Ledger and calls nothing else for the budget answer, so there is no second constant; a test derives the admit boundary as landingReserve(cap) + cost_precompact_note and then sweeps every remaining from zero upward asserting affordsHarnessNote(r) == (decideCalls(...) == .fire), which a parallel mechanism would fail; and the note turn passes the same shared run_budget instance, so it is counted in the pool where it is spent.

The token half (note_reserve_tokens = 2_000, checked against provider.context) is the direct analogue of Codex's buffer and is a gate, not a second call reserve.

Persistence: a session-scoped file, not playbook items

The issue suggested playbook items with source=session. Rejected on four structural grounds:

  1. The playbook is project-scoped and deliberately cross-sessionblockNow reads one .graff/playbook.jsonl from any session in the repo. A note about the refactor in flight is noise tomorrow, and source=session would force every existing reader to learn a session filter.
  2. playbook.max_text is 240 bytes per item with 2048/1024-byte block caps. Subgoal plus anchors plus decisions plus dead ends is 1–2k on its own.
  3. Every item rides every subagent/workflow brief via rideBrief. Pre-compaction notes-to-self: reserve buffer tokens so the agent writes durable state before context rollover (Codex pattern) #391 requires the opposite.
  4. Items are permanent until retired by id; a note is superseded wholesale by the next one.

So: .graff/notes/<session>.notes.jsonl, injected into the root system prompt beside HARD CONSTRAINTS. What #383's mechanism did contribute, unchanged: append-only JSONL written whole-or-not-at-all, replayed deterministically, and assembled from the file at injection time rather than from conversation memory. That single property is why it survives compaction with no extra machinery — there is no in-context copy for a summarizer to paraphrase — and the system prompt is re-sent verbatim every request.

Verification

zig build test1058/1058, exit 0 (baseline 1043, +15). A deliberate break confirmed the suite goes red before restoring, so the new modules genuinely compile in. scripts/eval-tier1.sh green, reachability at 1021 declared tests.

Covered: fires exactly once when compaction is imminent, and the same history_rewrites generation never buys a second note (so it survives #379's retry loop) while the next generation does; never fires for a subagent, asserted through the production entry point with provider/client left undefined, so a pass proves no request was attempted; budget refusal at the exact landingReserve + 1 boundary plus the full sweep; token-headroom refusal at the buffer boundary; a real file round trip in a scratch cwd with byte-identical readback, sessions not reading each other's notes, newest superseding, and torn tails or junk lines costing at most their own record; survives compaction and re-injects verbatim with the entire conversation replaced by a summary mentioning none of it; graceful degradation on empty/whitespace/"none"/path-escaping session names.

src/agent_compact.zig ends at 569 (+5: one import, three comment, one call). With #445 and #440 on top that lands near 588/600.

Not covered

The .fire branch issues a live provider request, which no unit test can drive, so the model call, the reply text, and the compact() call site firing are uncovered. Every refusal path is exercised through the real entry point, and the request is built from shapes already proven elsewhere (playbook_reflect.askModel's throwaway tool-less agent, compact()'s own history clone). compaction_request = true is set on the note agent so applyOverflowRecovery's early-out stops the note turn recursing into emergencyTrim mid-compaction — that reasoning is read off the source, not exercised by a test.

Integration note: this branch and #459 (#411) independently created a src/compact_note.zig for different purposes. They collide by filename and will be disambiguated when the two are integrated.

justrach and others added 2 commits August 6, 2026 19:03
Compaction hands the conversation to a summarizer, and a summarizer optimizes
for a readable account of what happened. The things a working agent cannot
cheaply re-derive - the exact line it was editing, the approach it already
ruled out, why it picked B over A - are precisely what a summary drops as
uninteresting. Codex solves this by reserving buffer tokens and firing one
prompt right before rollover; this is that, on graff's substrate.

When compaction is imminent the root now spends one bounded, tool-less turn
writing a note to its future self (subgoal, file:line anchors, decisions, dead
ends), stored outside the conversation and re-injected verbatim after the
history it describes is gone.

BUDGET: one ledger, not two. #390's landing reserve (phase_budget.Ledger) is
extended with cost_precompact_note and Ledger.affordsHarnessNote rather than
given a sibling. The note is the JUNIOR liability on that ledger - narration
is mandatory and owns the reserve, the note is optional and must fit ON TOP of
it, the same P3 gate judges pass. A second reserve would be a second ledger,
and two ledgers each holding back "the last call" double-count the same pool,
which is the bug #390 was filed for. The call then goes through the shared
RunBudget like any other, so it is counted where it is spent. A token buffer
(note_reserve_tokens) is the second half of the gate, checked against the
window. A test derives the call boundary from landingReserve() itself, so a
parallel reserve would move it and fail.

PERSISTENCE: a session-scoped append-only store (.graff/notes/<session>.notes.
jsonl) composed into the ROOT system prompt beside HARD CONSTRAINTS, not
playbook items with source=session. The playbook is project-scoped and
cross-session by design, caps items at 240 bytes, rides every subagent brief
through rideBrief, and keeps items until a user retires one by id - all four
are wrong for working state that is superseded by the next note and must never
reach a worker. What is borrowed is the MECHANISM, exactly: append-only JSONL
written whole-or-not-at-all, replayed deterministically, and assembled from the
file at injection time rather than from conversation memory. That last property
is the whole feature - there is no in-context copy for the next compaction to
paraphrase away, and the system prompt is re-sent verbatim on every request.

Never fires for a subagent (checked first, before anything is measured), at
most once per history generation however often compaction is retried (#379's
loop would otherwise buy a note per lap), and every refusal is a named skip:
compact() ignores the result, so a failed, empty or unwritable note costs
nothing but the note.

Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
CI's codex PTY scenario caught the note turn. Investigating it turned up a
real gap and corrected a wrong assumption about where the boundary is.

THE GAP, now closed. #391 is about a PLANNED rollover: context is filling, so
reserve a buffer and spend one call before the window turns over. It is not
about a rescue. Once the provider has rejected a request for exceeding the
window (last_request_context_overflow), or the meter is at the destructive-
recovery boundary where compactOrRecover may drop real history
(Provider.nearContextLimit, 95%), the session has demonstrably run out of room
and a note is the last thing it can afford. decideRoom becomes decideContext
and refuses those outright, BEFORE the token-buffer question — which would
have said yes, because at 95% of a 200k window there are still 10k tokens of
nominal room. contextOf reads the same two signals compactOrRecover uses to
authorize destructive trimming, so the note fires exactly when compaction is
scheduled and never when it is damage control. Two unit tests, one pure and
one through maybeWrite with a live provider.

THE ASSUMPTION, corrected with evidence. run_midturn_compaction_scenario is
NOT the recovery path. test-pty-codex-ws.py sets the server meter to 90% of
the window and codex_ws_test.py says why: "Cross compact@ (80%) but stay below
the destructive recovery boundary (95%)." runTurn's mid-turn gate is
inputOverCompactThreshold (80%), and trim_on_fail is false there. So it is the
planned rollover, and the single most representative instance of #391 in the
tree — a long tool-loop turn crossing the threshold mid-flight, the failure
mode the issue was filed over. Suppressing the note there would suppress it on
the main path.

So the scenario is taught the new shape rather than the note suppressed, and
the #195 WS invariant is CHECKED rather than argued: the note turn is a quiet
tool-less SSE request that opens no WebSocket (connection_id is None), runs
inside runTurn's existing closeCodexWs bracket against a throwaway clone of
history, and leaves the choreography untouched — ws_connections stays 2, the
post-compaction turn still re-anchors on a fresh socket, and no request
carries previous_response_id. Verified by running it, not by reasoning.

Both compaction fixtures are re-keyed on the request's last USER TURN instead
of its ordinal. An ordinal-keyed fixture re-targets silently when compaction
gains a step: the summary reply landed on the note turn, the note reply became
the handoff summary, and midturn still went green while proving something
else. The transactional scenario failed outright for the same reason.

And the scenario now pins #391 end to end on the wire, which no unit test can
reach: the note is absent from `instructions` before it is written, present
VERBATIM in every request after it, and absent from `input` on the
post-compaction turn — state, not conversation, with nothing in the history
for the next compaction to paraphrase away. Confirmed load-bearing by
disabling the injection and watching it fail.

Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
@justrach
justrach changed the base branch from main to release/0.0.242 August 6, 2026 12:32
@justrach
justrach merged commit 4d14719 into release/0.0.242 Aug 6, 2026
6 checks passed
@justrach
justrach deleted the feat/391-pre-compaction-notes branch August 6, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pre-compaction notes-to-self: reserve buffer tokens so the agent writes durable state before context rollover (Codex pattern)

1 participant