fix(codex): context meter tracks the full-history resend + recover from context-window rejections (#174) - #175
Merged
Merged
Conversation
…om context-window rejections (#174) Three-part fix for the wedged-session failure (meter reads 95k/270k while the backend rejects every request for exceeding the window): - recordUsageResponses takes max(server total, full-input estimate): the WS-chained total_tokens tracks a conversation the server has pruned (prior-turn reasoning is discarded from chained context), while every turn's FIRST request resends the full local history including all retained encrypted reasoning items. The meter now reflects what that resend costs (serialized input bytes / 4 via a counting discard writer, zero-alloc), so compact@80% fires before the wall — the same correction codex CLI applies (ContextManager get_non_last_reasoning_items_tokens). - 'exceeds the context window' rejections pin the meter to the window: the rejected request never returns usage to correct a stale meter, so without this the ApiError compact-and-recover gate can never pass and the session wedges permanently (every retry resends the same oversized history). - compact() drops prior-turn Responses reasoning items before the summary request (dropPriorTurnReasoning): the backend discards them from chained context anyway, and they dominate resend bloat on long high-effort sessions — this keeps the compaction request itself under the wall it is escaping. Current-turn reasoning stays (required between a function_call and its output). Verified against a scripted Responses mock (fat reasoning items, chained usage lying at total_tokens=100): meter grows 100 -> 10k over 6 turns instead of sitting at 100; a mid-session context rejection compacts and the session continues (history collapse visible as context_tokens 4115 -> 100); with the mock rejecting EVERY oversized body, every rejection now recovers instead of wedging. Unit tests for the estimator and the reasoning trim. Closes #174 Co-Authored-By: blackfloofie <265516171+blackfloofie@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug (#174)
A long gpt-5.6-sol session at Extra high reads
95k/270k ctx (35% · compact@216k)and then every request dies withYour input exceeds the context window of this model— permanently wedged.The meter (
last_context_tokens) copies the WS-chainedusage.total_tokens, which tracks a conversation the server has pruned (prior-turn reasoning is discarded from chained context). ButrunTurncloses the WS per turn, so every turn's first request resends the full local history — including every retained encrypted reasoning item. That resend is what the backend measures, it can be ~3x the chained number, and it's the request that gets rejected — so no corrective usage ever arrives, the meter never crosses compact@216k, and neither the pre-turn compaction nor the ApiError recovery ever fires.codex CLI avoids this by correcting its meter with an estimate of reasoning items the server accounting excluded (
ContextManager::get_non_last_reasoning_items_tokens, confirmed against openai/codex).The fix
recordUsageResponsestakesmax(server total, full-input estimate)— serializedinputbytes / 4 via a zero-alloc counting discard writer. compact@80% now fires before the wall.exceeds the context windowrejection pins the meter to the window, so the existing ApiError compact-and-recover path engages — rescues already-wedged sessions on their next input.compact()first drops Responses reasoning items older than the last user message (dropPriorTurnReasoning) — the backend discards those from chained context anyway, and they dominate resend bloat — so the compaction request itself fits under the wall it's escaping. Current-turn reasoning stays (required between afunction_calland its output).Verification (scripted Responses mock: fat reasoning items, chained usage lying at
total_tokens=100)context_tokens4115 → 100)Unit tests for the estimator and the reasoning trim;
zig build+zig build testclean.Closes #174. Related: #165 (this is the accounting consequence of the delta transport).