feat(repl): /btw — a tool-less side question over live context, never persisted (#415) - #466
Merged
Merged
Conversation
A throwaway question answered from the LIVE conversation with tools disabled, rendered once, and then dropped. "what did that error actually mean?" is a question ABOUT the work in progress, not a step of it, and asking it the ordinary way costs the session two permanent messages plus their re-send on every turn after. The turn runs on a throwaway agent over a container-deep CLONE of the root's history in an arena that dies with the call - compact_note_glue's shape, for its reason: send-time normalization mutates `messages` in place, so only a clone may be handed to it. `root.messages` is read once, to clone, and written never. That is what keeps it out of BOTH persistence paths, which are not the same path: the session file is a rewrite of `root.messages`, while the append-only transcript (#441) is an OBSERVATION of it taken inside session.queueSave - and being append-only, a line written there could never be taken back. Neither runs here, and neither could see the exchange if it did: the side agent's messages live in an arena the root holds no pointer to, and its `sub = true` makes `record` refuse it at the first line. It is still billed. The request goes through Agent.request like any other, so recordUsage/recordCost bank it in the same `pricing.g_cost` the [usage] footer and /cost read. Not keeping an exchange is not a reason to hide what it cost. Tools are off by construction: `request(null)` omits the `tools` field entirely rather than sending an empty array, for all three wire formats, and `text_only` keeps it tool-less if it is ever driven through runTurn. Single-shot, deliberately - the issue names multi-turn side threads a possible v2, and there is no side history here to continue. Both halves ship: the `/btw` slash command and a `{"type":"btw"}` --json control for the GUI, whose answer event carries `persisted:false` so a client mirroring the stream can tell it apart from a real turn. mainloop.zig was at the 600-line ceiling, so the JSON controls that never become a turn moved to json_controls.zig, taking the per-request tool knobs with them; mainloop ends up three lines shorter than it started. Proof, not assertion. The unit tests drive the real `build` and compare the session file and the transcript byte-for-byte across the exchange; a deliberate leak was planted and the transcript comparison itself is what caught it. A tier-2 case then runs the real binary against the scripted model and checks the request AFTER the side question: it must carry no trace of it, which is the only way it could reach either file. That case also goes red when the leak is planted. Co-Authored-By: Codegraff <blackfloofie@codegraff.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #415. A throwaway question answered from the live conversation with tools disabled, rendered but never added to the session. Single-shot only — the issue is explicit that multi-turn side threads are a v2 and "single-shot is the 90% win".
Proving "never persisted", which is the whole feature
There are now two persistence paths, not one: the session file, and the append-only transcript that landed in this same release (#441). A "never persisted" feature that leaked into the transcript would be wrong in exactly the way that is hard to notice, so this was proven three independent ways:
saveSessionTo+flushSavesrun for real against a tmp dir; the.session.jsonand.transcript.jsonlare snapshotted, the side exchange runs in its own arena, both files are saved again and compared withexpectEqualStrings, pluslineCount()andsession_writer.stats().writesunchanged. It also asserts neither file ever contained the question or answer text — a byte comparison alone would still pass if the first save had leaked.self.messages.append(question)) was planted insidebuild, and the failure was confirmed to be the transcript byte comparison itself, not an incidental count assertion. The assertions were then reordered so the transcript is checked first, precisely because it is the append-only path that cannot be undone.btw-side-question-leaves-no-trace: prompt →{"type":"btw"}→ second prompt against the scripted model. Asserts the btw request carries the prior context and the question and lacks"tools", and that the request after it carries no trace of either — that request being the only route toroot.messages, which is the only route to both files.Design:
compact_note_glue, notreview.zigreview.zigswaps the root's own history out and back, so it keeps using the root agent — the wrong shape when the requirement is that the root must not be touched at all.compact_note_glue.askModelalready does throwaway agent + container-deep clone of root history + root provider + no tools, which is/btwalmost exactly.Tools are off via
request(null), which omits the key entirely for all three wire formats;text_only = truekeeps it tool-less if anyone later routes it throughrunTurn. Billing needs no code — going throughAgent.requestbanks intopricing.g_costautomatically, and the test asserts the delta lands in the tally the[usage]footer renders, since the issue is explicit that the cost is real and should be visible.--jsoncontrol included{"type":"btw","text":"…"}→{"type":"btw","ok":true,"text":…,"persisted":false}. The explicitpersisted:falselets a client mirroring the stream tell it apart from a real turn.mainloop.zigwas at exactly 600, so rather than golf it, the per-request tool knobs (maxToolCalls/dedupeToolCalls) moved into a newjson_controls.zigbeside the btw control — a cohesive "control fields applied inline, never a turn" module. That leftmainloop.zigat 597, three lines shorter than it was found.A reachability catch worth repeating
eval-tier1 --only reachcaught thatjson_controls.zig's tests were not compiled in, despitemainloop.zigimporting it —mainloopis reached frommain(), not from the test root's analysis, so its imports are not pulled in for tests. Fixed with atest_hooks.zigentry; the count moved 1121 → 1123 as a result. Without the reach check those two tests would have reported green while never running.Verification
zig build test→ 1123/1123, +7 from the 1116 baseline.scripts/eval-tier1.shgreen.scripts/eval-tier2.pygreen at 41/41, including every pre-existing case, so the mainloop extraction broke nothing.Also smoke-tested through the real TUI path against the mock: empty
/btwprints the usage hint, a real one prints the answer plus a "not added to this session" footer, the request goes out with notoolskey, and neither on-disk file gains a trace.Not covered
The network request and the model's reply cannot be unit-driven, so
ask's request line rests on tier-2 plus the shape borrowed wholesale fromcompact_note_glue. Not exercised against a real provider or on the Responses/codex WS path (the tier-2 mock is the OpenAI wire format);defer agent.closeCodexWs()is in place so the throwaway cannot leak a socket, but that path is untested. The GUI does not yet consume the new event.