feat(compact): reserve a buffer and write notes-to-self before context rollover (#391) - #460
Merged
Merged
Conversation
Compaction hands the conversation to a summarizer, and a summarizer optimizes for a readable account of what happened. The things a working agent cannot cheaply re-derive - the exact line it was editing, the approach it already ruled out, why it picked B over A - are precisely what a summary drops as uninteresting. Codex solves this by reserving buffer tokens and firing one prompt right before rollover; this is that, on graff's substrate. When compaction is imminent the root now spends one bounded, tool-less turn writing a note to its future self (subgoal, file:line anchors, decisions, dead ends), stored outside the conversation and re-injected verbatim after the history it describes is gone. BUDGET: one ledger, not two. #390's landing reserve (phase_budget.Ledger) is extended with cost_precompact_note and Ledger.affordsHarnessNote rather than given a sibling. The note is the JUNIOR liability on that ledger - narration is mandatory and owns the reserve, the note is optional and must fit ON TOP of it, the same P3 gate judges pass. A second reserve would be a second ledger, and two ledgers each holding back "the last call" double-count the same pool, which is the bug #390 was filed for. The call then goes through the shared RunBudget like any other, so it is counted where it is spent. A token buffer (note_reserve_tokens) is the second half of the gate, checked against the window. A test derives the call boundary from landingReserve() itself, so a parallel reserve would move it and fail. PERSISTENCE: a session-scoped append-only store (.graff/notes/<session>.notes. jsonl) composed into the ROOT system prompt beside HARD CONSTRAINTS, not playbook items with source=session. The playbook is project-scoped and cross-session by design, caps items at 240 bytes, rides every subagent brief through rideBrief, and keeps items until a user retires one by id - all four are wrong for working state that is superseded by the next note and must never reach a worker. What is borrowed is the MECHANISM, exactly: append-only JSONL written whole-or-not-at-all, replayed deterministically, and assembled from the file at injection time rather than from conversation memory. That last property is the whole feature - there is no in-context copy for the next compaction to paraphrase away, and the system prompt is re-sent verbatim on every request. Never fires for a subagent (checked first, before anything is measured), at most once per history generation however often compaction is retried (#379's loop would otherwise buy a note per lap), and every refusal is a named skip: compact() ignores the result, so a failed, empty or unwritable note costs nothing but the note. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
CI's codex PTY scenario caught the note turn. Investigating it turned up a real gap and corrected a wrong assumption about where the boundary is. THE GAP, now closed. #391 is about a PLANNED rollover: context is filling, so reserve a buffer and spend one call before the window turns over. It is not about a rescue. Once the provider has rejected a request for exceeding the window (last_request_context_overflow), or the meter is at the destructive- recovery boundary where compactOrRecover may drop real history (Provider.nearContextLimit, 95%), the session has demonstrably run out of room and a note is the last thing it can afford. decideRoom becomes decideContext and refuses those outright, BEFORE the token-buffer question — which would have said yes, because at 95% of a 200k window there are still 10k tokens of nominal room. contextOf reads the same two signals compactOrRecover uses to authorize destructive trimming, so the note fires exactly when compaction is scheduled and never when it is damage control. Two unit tests, one pure and one through maybeWrite with a live provider. THE ASSUMPTION, corrected with evidence. run_midturn_compaction_scenario is NOT the recovery path. test-pty-codex-ws.py sets the server meter to 90% of the window and codex_ws_test.py says why: "Cross compact@ (80%) but stay below the destructive recovery boundary (95%)." runTurn's mid-turn gate is inputOverCompactThreshold (80%), and trim_on_fail is false there. So it is the planned rollover, and the single most representative instance of #391 in the tree — a long tool-loop turn crossing the threshold mid-flight, the failure mode the issue was filed over. Suppressing the note there would suppress it on the main path. So the scenario is taught the new shape rather than the note suppressed, and the #195 WS invariant is CHECKED rather than argued: the note turn is a quiet tool-less SSE request that opens no WebSocket (connection_id is None), runs inside runTurn's existing closeCodexWs bracket against a throwaway clone of history, and leaves the choreography untouched — ws_connections stays 2, the post-compaction turn still re-anchors on a fresh socket, and no request carries previous_response_id. Verified by running it, not by reasoning. Both compaction fixtures are re-keyed on the request's last USER TURN instead of its ordinal. An ordinal-keyed fixture re-targets silently when compaction gains a step: the summary reply landed on the note turn, the note reply became the handoff summary, and midturn still went green while proving something else. The transactional scenario failed outright for the same reason. And the scenario now pins #391 end to end on the wire, which no unit test can reach: the note is absent from `instructions` before it is written, present VERBATIM in every request after it, and absent from `input` on the post-compaction turn — state, not conversation, with nothing in the history for the next compaction to paraphrase away. Confirmed load-bearing by disabling the injection and watching it fail. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #391. The Codex
auto_compact_fallback_buffer_tokens/auto_compact_fallback_promptpattern: the agent writes what its future self needs before compaction, instead of losing working state silently.The reserve shares #390's ledger, it does not add a second one
This was the issue's hardest constraint, so it gets the detail. The reservation lives in
src/phase_budget.zig, the ledger over the sharedRunBudgetpool where #390's landing reserve already lives (landingReserve,Ledger.reserve; its hard half is thedepth > 0 and remaining() <= 1guard inrun_budget.zig:97). The addition iscost_precompact_note: u64 = 1plusLedger.affordsHarnessNote(remaining), which is literallyfits(remaining, cost_precompact_note)— the same "must fit on top of the reserve" predicatesubagent.scoreVariantsuses for judges.The note is deliberately the junior liability: narration is mandatory and owns the reserve, the note is optional and must clear it. A separate reserve would have been a second ledger over one pool, double-counting the last call.
Confirmed three ways rather than asserted:
decideCallsconstructs aphase_budget.Ledgerand calls nothing else for the budget answer, so there is no second constant; a test derives the admit boundary aslandingReserve(cap) + cost_precompact_noteand then sweeps everyremainingfrom zero upward assertingaffordsHarnessNote(r) == (decideCalls(...) == .fire), which a parallel mechanism would fail; and the note turn passes the same sharedrun_budgetinstance, so it is counted in the pool where it is spent.The token half (
note_reserve_tokens = 2_000, checked againstprovider.context) is the direct analogue of Codex's buffer and is a gate, not a second call reserve.Persistence: a session-scoped file, not playbook items
The issue suggested playbook items with
source=session. Rejected on four structural grounds:blockNowreads one.graff/playbook.jsonlfrom any session in the repo. A note about the refactor in flight is noise tomorrow, andsource=sessionwould force every existing reader to learn a session filter.playbook.max_textis 240 bytes per item with 2048/1024-byte block caps. Subgoal plus anchors plus decisions plus dead ends is 1–2k on its own.rideBrief. Pre-compaction notes-to-self: reserve buffer tokens so the agent writes durable state before context rollover (Codex pattern) #391 requires the opposite.So:
.graff/notes/<session>.notes.jsonl, injected into the root system prompt beside HARD CONSTRAINTS. What #383's mechanism did contribute, unchanged: append-only JSONL written whole-or-not-at-all, replayed deterministically, and assembled from the file at injection time rather than from conversation memory. That single property is why it survives compaction with no extra machinery — there is no in-context copy for a summarizer to paraphrase — and the system prompt is re-sent verbatim every request.Verification
zig build test→ 1058/1058, exit 0 (baseline 1043, +15). A deliberate break confirmed the suite goes red before restoring, so the new modules genuinely compile in.scripts/eval-tier1.shgreen, reachability at 1021 declared tests.Covered: fires exactly once when compaction is imminent, and the same
history_rewritesgeneration never buys a second note (so it survives #379's retry loop) while the next generation does; never fires for a subagent, asserted through the production entry point withprovider/clientleftundefined, so a pass proves no request was attempted; budget refusal at the exactlandingReserve + 1boundary plus the full sweep; token-headroom refusal at the buffer boundary; a real file round trip in a scratch cwd with byte-identical readback, sessions not reading each other's notes, newest superseding, and torn tails or junk lines costing at most their own record; survives compaction and re-injects verbatim with the entire conversation replaced by a summary mentioning none of it; graceful degradation on empty/whitespace/"none"/path-escaping session names.src/agent_compact.zigends at 569 (+5: one import, three comment, one call). With #445 and #440 on top that lands near 588/600.Not covered
The
.firebranch issues a live provider request, which no unit test can drive, so the model call, the reply text, and thecompact()call site firing are uncovered. Every refusal path is exercised through the real entry point, and the request is built from shapes already proven elsewhere (playbook_reflect.askModel's throwaway tool-less agent,compact()'s own history clone).compaction_request = trueis set on the note agent soapplyOverflowRecovery's early-out stops the note turn recursing intoemergencyTrimmid-compaction — that reasoning is read off the source, not exercised by a test.Integration note: this branch and #459 (#411) independently created a
src/compact_note.zigfor different purposes. They collide by filename and will be disambiguated when the two are integrated.