feat(context): oversized tool outputs spill to a session artifact (#409) - #436
Merged
Conversation
The per-output cap (#193/#196) destroyed the elided bytes: the model's only recovery was to re-run the tool and guess a better slice. Now, when the agent has a durable session, the FULL output is written to `.graff/sessions/<session>/artifacts/tool-<n>.txt` before the cap shrinks it, and the marker cites the absolute path and the byte count, so the next turn can read or grep exactly what it needs. Same call site and same safety class as #196 (no WS-close bracketing): the cap still only shrinks strings in place. A subagent has no persisted history, so it has no durable session and keeps the plain truncation, as does any process that never wired a sink (every unit test). Growth is bounded twice: `session_cap_bytes` (64 MiB, all-or-nothing per artifact so a marker can never lie about its byte count), and reclamation of the artifact dirs whose `<session>.session.json` is gone — the session file is the ground truth for "this session was deleted", so an rm, the AI-title rename and /new all reclaim what they left behind. The sweep runs once, at the first spill, so a run that never spills does no extra I/O. The cap's truncation primitives move to the new module with it: the spill has to happen inside truncateStrField, the one place still holding the pre-truncation string, and agent_compact.zig sits at the 600-line cap. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
scripts/test-spill-artifact.py drives a real graff against the repo's scripted model (scripts/eval/mock_model.py on the lmstudio port) with a seeded session whose last tool output is over the per-output cap, and asserts all three halves of the claim: the request the harness sent carries the marker (absolute path + byte count) and NOT the elided needle, the artifact on disk holds the original bytes byte for byte, and a follow-up `bash` call against THE PATH THE MARKER CITED brings the needle back. Negative control: with the sink unwired the test fails with "no #409 marker" and no read-back. The marker's path is now resolved with realPathFile through the same dir handle the artifact was written with. The declared base (g_cwd_display) falls back to $PWD, which a caller that changed directory without exporting it gets wrong — and that produced a path that pointed at nothing, exactly what the e2e caught. The user-facing cap note now distinguishes the two outcomes: bytes elided but kept as an artifact, versus the pre-#409 destructive truncation (#202). Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
CHANGELOG entry for v0.0.240 and the README's context-management paragraph, which described the tool-result preview pointer but stopped at the send-time cap, where the bytes used to simply go. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
…ename (#409) Moving a renamed session's artifacts (or dropping them at the rename) would strand every path already handed to the model in that transcript. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
…an forge The marker embeds the artifact's REAL absolute path, and std.testing.tmpDir names its directory with 16 random base64-url characters. 'N' is in that alphabet, so "the needle is gone from the transcript" failed whenever the random name happened to contain one: 4 failures in 15 runs, measured identically on this integration AND on feat/prime-409-spill alone. Test-only; the assertion's intent is unchanged. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
…hs are not /-rooted (#409) The test matched 'are at /' and posix separators in the marker; on windows the resolved path is drive-rooted with backslashes and the windows CI job failed on exactly this test. The path is now extracted from the marker and asserted absolute via std.fs.path.isAbsolute, with the artifact name matched separator-free. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #409 — spill, don't truncate. The RLM 'context as variables' payoff without adopting a kernel: the per-result cap (#193/#201) no longer destroys the elided bytes.
Mechanism
When the session is durable, the full pre-truncation output is written to
.graff/sessions/<session>/artifacts/tool-<n>.txtand the elision marker cites the absolute path + byte count, telling the model to read/grep the slice it needs instead of re-running the tool. The spill lives insidetruncateStrField— the one place still holding the pre-truncation string — which forced a healthy split: the cap primitives moved fromagent_compact.zig(598→564) into the newtool_spill.zig.Safety rails: subagents (history never persisted) keep plain truncation; session names are traversal-validated; a 64 MiB per-session budget reserved atomically per whole artifact (a marker can never overstate what's on disk); a marker that can't fit inside the cap falls back to the short note so the cap never grows an output. Artifact dirs whose session file is gone are swept at the next spill (1h orphan grace for a concurrent pre-autosave graff; deliberately not hooked at rename, which would strand paths already cited in the live transcript). The artifact path resolves through the dir handle it was written with — the $PWD fallback produced a dead path in a chdir'd caller, and the E2E caught it.
Experiment
scripts/test-spill-artifact.py, CI-wired: a seeded session carries an over-cap output with a needle past the cap; asserts the artifact holds all 30,000 bytes byte-for-byte, the request carries the marker (and not the needle), the mock then reads the exact path the marker cited and the needle comes back in the next request. Negative control: sink unwired → test fails. 3× stable.Evidence
Tests 994 → 1000 (+6 exactly: durability gate, budget bounds incl. exact-fit/one-over/saturation, traversal names, real disk spill, sweep keep/reclaim). All 9 golden eval files PASS byte-identical. fmt + line-guard clean. Changelog under
v0.0.241 (unreleased).Noted for a possible follow-up: #163's
trimOldestToolOutputsstill truncates destructively — one field away from spilling too.Part of the prime-agent adoption batch.