Skip to content

context-overflow: perOutputCap × keep_recent=4 lets recent tool outputs wedge the window (hole in #193 per-output cap) #201

Description

@justrach

Symptom

The #193 follow-up (per-output cap + in-turn recovery, b227ef8) bounds any single
tool output before send and, on an over-window rejection, emergencyTrim /
trimOldestToolOutputs reclaims room and retries. But the trim keeps the
keep_recent = 4 most-recent tool outputs verbatim, and perOutputCap() is
window-proportional at context * 2 bytes (~50% of the window in estimated
tokens). So four recent large outputs can pin ~2x the window in content the
recovery cannot touch -> the retried request is still over-window -> hard wedge,
escapable only by a manual /compact.

This is a hole in what shipped in #193, not a new subsystem.

Evidence

  • src/provider.zig:97-100 - perOutputCap() = context * 2 bytes (~context/2 tokens per output)
  • src/agent_compact.zig:443,455 - trimOldestToolOutputs keeps 4 recent verbatim (keep_recent = 4)

Invariant violated

keep_recent * perOutputCap must be <= what emergencyTrim can reclaim.
Currently 4 * ~50% window = ~200% window, so recovery can never bring a
recent-output burst back under the wall.

Proposed fix

Couple the two so the recent-output floor stays reclaimable:

  • Lower perOutputCap() to ~1/8 of the window (context / 2 bytes) so 4 recent
    outputs ~= 50% of the window, leaving headroom for the trimmed remainder +
    system prompt to fit on retry.
  • Add an absolute ceiling (~256 KB, codex-style fixed cap) so huge-window models
    still bound one pathological single result.
  • Optionally drop keep_recent 4 -> 2 for more margin.
  • Sanity-check the constant against the smallest supported local window so the
    cap never exceeds the window itself.

openai/codex comparison

openai/codex caps every exec/function output at an absolute ceiling
(default 10k tokens/bytes, format_exec_output_str -> formatted_truncate_text,
per-model TruncationPolicy) with no recency exemption. That fixed cap is why
one fat result rarely wedges it. We adapt (keep the window-proportional cap for
large windows, add the absolute ceiling).

Refs

#193 (parent), #192

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:p1High priority

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions