Skip to content

recognize litellm maximum-context rejections - #2129

Merged
snimu merged 5 commits into
mainfrom
fix/litellm-context-overflow
Sep 9, 2026
Merged

snimu merged 5 commits into
mainfrom
fix/litellm-context-overflow

Conversation

@kevinjosethomas

@kevinjosethomas kevinjosethomas commented Sep 8, 2026 •

Copy link
Copy Markdown
Member
  • recognizes litellm maximum-context rejections so existing overflow compaction runs before retry; addresses eng-6011, reported in discussion #2088.
  • validated head a2131be2773119e7ee1fe6e8cc764a95242e7803 against baseline 4ec05a0969825a32261ffed43ab5740071a73ae6 in prime sandbox xboin1vp7ctxmzop9sjxksuh (node:24-bookworm, linux x64, node 24.18.0, npm 11.16.0, 2 cpu/4 gb ram/10 gb disk): the compiled, packed, and installed ai package changed the reported classification from false to true, with four controls unchanged; sandbox deleted after collecting logs.
  • passed 18 classifier tests, 74 faux-provider compaction/retry tests, and npm run check locally and in the sandbox; identical baseline fixtures failed only the new rejection cases, including recovery, while controls passed; no live litellm/vllm or paid inference calls; code ci passed.

Linear: ENG-6011


Note

Medium Risk
Changes shared overflow classification used by agent compaction/retry paths; misclassification could compact on rate limits or skip compaction on overflow, though new negative tests reduce that risk.

Overview
LiteLLM context rejections (when input plus requested completion exceed the model limit) are now treated as context overflow, so the coding agent runs overflow compaction and continues instead of treating the failure as a generic error or retrying the same oversized request.

The packages/ai overflow classifier adds a pattern for messages like "exceeds the model's maximum context length", with unit tests for case-insensitive matching and guards so rate limits, token quotas, and unrelated LiteLLM errors are not misclassified. A coding-agent regression test confirms compaction and recovery for the real LiteLLM error string (and still retries rate-limit errors without compacting).

Contributor workflow: the PR template and Linear ticket CI now accept ENG- tickets (in addition to RES-), with updated guidance for Engineering vs Research boards.

Reviewed by Cursor Bugbot for commit a65ce8e. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Fix isContextOverflow to recognize LiteLLM maximum-context rejections

  • Adds a case-insensitive pattern in overflow.ts that matches LiteLLM messages stating the requested token count exceeds the model's maximum context length.
  • LiteLLM context-limit rejections now trigger automatic overflow compaction and recovery, rather than being missed or retried unchanged.
  • Adds regression tests in overflow.test.ts and 6011-litellm-context-overflow.test.ts covering positive detection, negative cases (token rate limits, per-minute quotas, unsupported parameters), compaction-and-recovery flow, and confirmation that token-rate-limit errors stay on the retry path.
  • Risk: if a LiteLLM rejection message format differs from the added pattern, isContextOverflow will not detect it; the pattern only matches the documented "maximum context" wording.
📊 Macroscope summarized a65ce8e. 2 files reviewed, 2 issues evaluated, 2 issues filtered, 0 comments posted

🗂️ Filtered Issues

.github/workflows/linear-ticket.yml — 0 comments posted, 2 evaluated, 2 filtered
  • line 23: The Linear-link alternative is not anchored to a host or even a URL boundary. A non-Linear string such as https://notlinear.app/team/issue/ENG-123 (or a path containing linear.app/...) contains the matched substring linear.app/team/issue/ENG-123, so it passes the gate despite not linking to Linear. Require an appropriate scheme/host boundary before linear.app when recognizing URL references. [ Out of scope (post-validation triage) ]
  • line 23: The URL alternative ends immediately after \d+, so it accepts only the numeric prefix of a malformed reference. For example, a PR body containing https://linear.app/team/issue/ENG-123not-a-ticket makes ticket.test(text) succeed even though the purported issue ID has trailing word characters; the gate is therefore bypassed without a valid ticket or No-Ticket opt-out. Add a trailing word boundary (or an equivalent URL/ID terminator) to this alternative. [ Out of scope (post-validation triage) ]

@kevinjosethomas
kevinjosethomas marked this pull request as ready for review September 9, 2026 00:36
@snimu
snimu merged commit 55ade48 into main Sep 9, 2026
24 checks passed
@snimu
snimu deleted the fix/litellm-context-overflow branch September 9, 2026 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants