feat(tools): tool results return handles by default above a size threshold (#440) - #451
Merged
Merged
Conversation
## What changed `src/tool_handle.zig` is the one size contract for a tool result. At or below the threshold (4 KiB, `GRAFF_TOOL_HANDLE_BYTES`) a result reaches the model exactly as the tool produced it. Above it, the complete bytes are written under `.graff/tool-results/` and the model gets a handle instead: a bounded preview, that file's ABSOLUTE path, the byte count, and a one-line shape hint measured from the payload (line count, or the top-level keys of JSON, only after the document scans clean end to end). Meta tools never touch it — runTools applies it to the external half of a batch. This collapses three caps that used to sit in series: - `agent_tools.persistToolResult` + `toolPreviewText` (2000 chars + a bare path) are REMOVED. They were the handle idea without the information that makes a handle usable: the model was told where the bytes were and nothing about what was in them, so it could not decide what to ask for without re-running the tool. - `tools.bash_stdout_cap` is DEMOTED from a context cap to a capture ceiling and raised 128 KiB -> 1 MiB, and `codedb`'s 64 KiB result truncation is REMOVED outright (the const survives as the compact-read capture bound). Both existed to protect context by destroying bytes; the handle protects context harder, non-destructively. At 128 KiB the handle would have lied for exactly the case that motivated the issue — a 168 KB log, a quarter of it already gone before anything could be written down. - `capOversizedToolOutputs` / `perOutputCap` / the #409 spill are KEPT, and narrowed to a backstop. `tool_handle.effectiveThreshold` clamps the tool-time threshold at or below `perOutputCap()`, so a result this process produced can no longer reach the send-time pass oversized. What still can is history this process did not produce — a session resumed from a pre-#440 build — which is precisely where deleting the pass would mean destroying bytes instead. A capability-gated prompt segment (`prompt_text.tool_handle_note`, gate `.local_tools`) tells the model the contract exists. It ships only when the tools that produce a handle and the tools that can open one are both present: under `--no-local-tools` a path is not something the session can act on. ## Why - Problem/failure mode: the send-time spill (#409) was unreachable for the local toolset. Every `execTool` result was already reduced to 2000 chars at tool time, so nothing ever arrived at the ~136 KB send-time threshold — in 12 live runs against a 168 KB log, including a forced `cat`, it never fired once. Meanwhile the bytes that mattered had already been destroyed twice over, by the 128 KiB bash cap and the 64 KiB codedb cap. - Reason for this approach: a cap only rescues bytes at the moment of overflow. Handles-by-default means a large result never occupies context in the first place, and one result costs one threshold however large it is. The filesystem is the namespace and bash is the REPL, both of which graff already has, so this needs no kernel and no new tool. - Constraints or trade-offs: the preview budget per oversized result roughly doubles (2000 -> 4096 bytes), which is the price of a head worth reading next to a marker worth acting on; it is constant, where the 128 KiB / 64 KiB / 136 KiB numbers it replaces were not. A handle is written whole or not at all, bounded by a per-process budget, because a partial one would make the byte count a lie. - Rejected alternatives: deleting the send-time cap as dead code. It is dead for anything runTools produced, and only for that; a resumed old session still carries outputs that never met this contract, and for those the alternative to the pass is an overflow. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
This was referenced Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #440. The generalization of #409, and the core of graff's answer to "context as variables" without adopting a Python kernel: the filesystem is the namespace, bash is the REPL, both of which graff already has.
The finding this fixes
A live A/B eval showed #409's spill path is effectively dead code for the local toolset. Three caps sat in series, and layer 2 reduced every
execToolresult to 2,000 chars, so layer 3's ~136 KB send-time threshold was unreachable for bash/read_file/webfetch/codedb. Across 12 runs against a 168 KB log, including a forcedcat, the spill never fired once.The contract
Above a threshold, a tool result returns, uniformly and at tool time: a bounded preview, a durable handle (absolute path), the byte count, and a one-line shape hint (line count for text, top-level keys for JSON). The model slices what it needs with
read_file/grep/bashinstead of carrying the payload turn after turn.What happened to the three caps
tool_preview_chars2000 +persistToolResulttool_handle.zig. It was the handle idea without the payload information that makes a handle actionable: path only, no size, no shape. Same on-disk namespace, so nothing is orphaned. Paths are now absolute, which also fixes a latent bug where a worktree-isolated subagent (#276) could never open the old relative path.bash_stdout_cap128 KiBrunCappedbuffers incrementally, so this is a ceiling, not an allocation.perOutputCap/capOversizedToolOutputs/tool_spilleffectiveThreshold(perOutputCap())clamps the tool-time threshold at or below the send-time cap, so the two are ordered by construction rather than stacked. What can still reach the backstop is a session resumed from a pre-#440 build, whose stored outputs never met this contract; deleting the pass would mean destroying those bytes.Threshold default is 4096 bytes, knob
GRAFF_TOOL_HANDLE_BYTES(0/unparseable ignored, so the contract cannot be silently disabled). Verified to sit above the tier-2 eval case whose 4001-byte outputs must stay whole to drive compaction. Prompt guidance ships as a capability-gated segment per #421, so it contributes zero bytes under--no-local-tools.Verification
zig build test→ 1048/1048, exit 0 (baseline 1043; +6 new, −1 removed, which proves the new module's tests are actually reachable).scripts/eval-tier1.shfully green.Beyond unit tests, this was driven end to end through the real harness against a mock Codex: a 20 KB
read_filecame back as a handle with preview ≤ 4096 bytes, an absolute path, the exact byte count, and shape hint1 lines, and the on-disk file was byte-for-byte equal to the payload.Behaviour changes worth knowing
[full tool result: … — inspect with read_file]breaks; the only in-repo consumer was updated..graff/tool-results/...relative.bashcommand emitting 128 KiB–1 MiB no longer reports[stdout truncated at 128 KB]; it yields a complete handle..graff/tool-results/can grow faster. Bounded by a new 64 MiB per-process budget with an honest truncation marker, but there is still no GC sweeper for that directory. Pre-existing gap, filed separately.Not measured: the 1 MiB capture ceiling's memory behaviour under real load is reasoned from incremental reads, not benchmarked.
src/session_run.zigsits at 599/600 lines, so the next change there needs an extraction first.