Skip to content

Trim #2161/#2164 tool-schema bloat; surface turn-level inference failures - #2168

Merged
mstomar125 merged 2 commits into
mainfrom
issue-2167-context-window
Aug 19, 2026
Merged

mstomar125 merged 2 commits into
mainfrom
issue-2167-context-window

Conversation

@mstomar125

Copy link
Copy Markdown
Contributor

Summary

Closes #2167.

The ai_chat_send_reaches_idle_with_no_stuck_processing_state integration test's underlying scenario was reported failing with Prompt uses 6032 tokens but context window is 5120 — the assembled system prompt for a fail-open (no skill matched) turn exceeding this machine's ADR-062-computed context window. This PR does the two things the issue's acceptance criteria call for.

1. Measured, not assumed

Reproduced the exact failing-test scenario and captured the real assembled prompt via NODESPACE_PROMPT_DUMP (the existing dev tool for this), then tokenized every tool-schema segment with the real locked model's tokenizer (not char-count estimates). On this machine the turn is a fail-open turn (no skill candidate clears the routing score gate for "Reply with exactly one word: OK"), so it gets the full unscoped tool surface: 6932 tokens = 713 (resident prose) + 6202 (all 13 tool schemas) + 17 (trailer). create_schema (1341 tok) and update_schema (1034 tok) — the two tools #2164 (friendly_name exposure) touched — together account for over a third of the tool-schema budget. declare_write_tool_fields (#2120/#2148/#2147) contributes zero extra tokens for this scenario, confirmed by direct inspection: no skill candidate clears the score gate, so no field declarations are generated.

(Note: on this dev machine, fit_n_ctx_to_budget grants the full configured 32768-token window given its RAM, so this specific run doesn't itself hit the 5120-token ceiling the issue reports for a more memory-constrained machine — the token-size finding and trim above are still real and machine-independent; the fix in part 2 below is the actual safety net for any machine where the window IS smaller.)

2. Trimmed, within the correct ADR-064 channel

  • create_schema's per-field name/friendlyName/description schema descriptions shortened to match update_schema's already-tighter phrasing for the same argument shape (previously two different-length explanations of the same thing).
  • update_schema.rename_fields's description tightened — same facts, roughly half the words. It was restating almost verbatim what the RENAME_VS_RELABEL skill rule already says.
  • RENAME_VS_RELABEL (skill guidance) cut back to only the procedural judgment call (which from/to shape a user's intent maps to); the mechanical explanation now lives solely in the tool schema, per ADR-064 rule 1. Regenerated packages/skill/SKILL.md from the trimmed rule via gen_skill_md.rs --write.
  • route_clarify's (Register a Stage-2-callable route_clarify tool #2161) description tightened slightly.
  • Along the way, fixed a pre-existing cargo clippy --all-targets -D clippy::unused-async failure in the same file (exec_route_clarify declared async with no .await in its body, from Register a Stage-2-callable route_clarify tool #2161) — this was blocking a clean clippy run independent of this issue, in code this PR already touches.

Net measured result for the scenario above: 6932 → 6810 tokens (122 saved, confirmed via the real tokenizer). Regenerated the prompt_assembly_snapshot goldens (#2119/#2120) to pin the new text — diffs reviewed, each one is exactly the intentional trim above.

3. Silent-failure fix (the issue's 3rd acceptance criterion)

local_agent_service.rs::run_ai_chat_turn previously logged a WARN and reset the node straight to "idle" on ANY inference failure (context overflow included), with no new assistant message — indistinguishable, to a caller polling get_node (or the frontend, which only renders user/assistant messages), from a turn still in flight. It now appends an assistant-role message naming the failure and reaches idle through the same path a normal reply uses (append_assistant_message), matching ADR-062's "refuse loudly, don't clamp silently" principle applied at the per-turn level, not just at model load. A genuine cancellation still resets silently (that's an intentional user action, not a failure).

Added failed_inference_turn_surfaces_a_visible_error_not_silent_idle as a regression guard (a FailingEngine stub returning InferenceError::ContextOverflow, asserting the node reaches idle WITH a new assistant message naming the failure).

Verification

  • cargo test -p nodespace-agent --lib: 522 passed
  • cargo test -p nodespace-daemon --lib: 153 passed (includes the new regression test)
  • cargo test -p nodespace-agent --test prompt_assembly_snapshot: 5/5 passed against regenerated goldens
  • cargo test -p nodespace-app --test ai_chat_send_to_idle_test ai_chat_send_reaches_idle_with_no_stuck_processing_state -- --test-threads=1: passed in isolation (no concurrent worktree builds)
  • cargo clippy --all-targets -- -D warnings -D clippy::unused-async: clean across the whole workspace
  • cargo fmt --check on every file this PR touches: clean
  • bun run test:all: full pass (frontend, skill, Rust workspace)
  • Full scripts/test-gate.ts (the real pre-push gate): ran clean end-to-end via git push, including the target ADR-048 suite

Test plan

  • Root cause of the prompt-size growth measured with real tokenizer numbers, not assumed
  • ai_chat_send_reaches_idle_with_no_stuck_processing_state passes reliably
  • Silent-failure behavior fixed with a regression test
  • bun run test:all and the full pre-push gate pass cleanly on a clean checkout

🤖 Generated with Claude Code

mstomar125 and others added 2 commits August 19, 2026 15:29
…ures

Measured the actual assembled prompt for the failing ai-chat scenario via
NODESPACE_PROMPT_DUMP + the real tokenizer: on this machine the turn's
full-tool-surface prompt is 6932 tokens (713 resident prose + 6202 across
all 13 fail-open tools + 17 trailer), with create_schema (1341) and
update_schema (1034) — the two tools #2164 touched — together accounting
for over a third of the tool-schema budget. declare_write_tool_fields
contributes zero extra tokens for this scenario (no skill candidate clears
the score gate, so field declarations never fire).

Trims applied within the tool-schema/skill-guidance channels (ADR-064):
- create_schema's per-field name/friendlyName/description descriptions
  shortened to match update_schema's already-tighter phrasing instead of
  carrying two different-length explanations of the same argument shape.
- update_schema's rename_fields description tightened (identical facts,
  half the words) — it was restating, almost verbatim, what the
  RENAME_VS_RELABEL skill rule also says.
- RENAME_VS_RELABEL (skill_rules.rs) cut back to the procedural judgment
  call (which shape the user's intent maps to); the mechanical "from"/"to"/
  "friendlyName" explanation now lives only in the tool schema, its correct
  channel. Regenerated packages/skill/SKILL.md from the trimmed rule.
- route_clarify's (#2161) description tightened slightly.
- Fixed a pre-existing `cargo clippy --all-targets -D clippy::unused-async`
  failure in the same file: exec_route_clarify was declared async with no
  .await in its body (from #2161); made it a plain sync fn.

Net: 6932 -> 6810 tokens for the measured scenario (122 saved, confirmed
via the real tokenizer, not estimated). The prompt_assembly_snapshot goldens
were regenerated to pin the new tool-schema/skill-guidance text.

Also fixes the silent-failure half of this issue: local_agent_service.rs's
run_ai_chat_turn logged a WARN and reset the node straight to "idle" on any
inference failure (including ContextOverflow) with no new assistant message
and no way for a polling caller or the frontend to distinguish a failed turn
from one still in flight. It now appends an assistant-role message naming
the failure and reaching idle through the same path a normal reply uses,
matching ADR-062's "refuse loudly, don't clamp silently" principle applied
at the per-turn level. Added
failed_inference_turn_surfaces_a_visible_error_not_silent_idle as a
regression guard.

Closes #2167.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… history tradeoff

Adversarial review of PR #2168 found no blockers. Two non-blocking items
addressed here:

- route_clarify's description had dropped "and nothing already said picks
  one" when trimmed — restored (a few tokens) since without it the text
  reads as license to clarify on any multi-match search result even when
  other conversational context already disambiguates.
- Documented, at the append_assistant_message call for a failed turn, why
  feeding the synthesized failure text back into the model's own future
  history as an ordinary assistant message is accepted rather than
  filtered/tagged: it can't trip the duplicate-write guard (empty
  completed_writes), unbounded growth is bounded by the existing
  maybe_summarize_history mechanism, and a model that sees its own stated
  failure has exactly the context needed to avoid blindly repeating it.

Also: the review noted this PR's first commit message states a net
6932 -> 6810 token result against the issue's reported 5120-token window
without reconciling that 6810 > 5120 — worth stating plainly here since
that commit's own text doesn't: the 5120 figure is this issue's reported
window on ITS reporting machine. On this machine, ADR-062's
`fit_n_ctx_to_budget` (packages/nlp-engine/src/chat/mod.rs) grants the
full configured 32768-token window given its total RAM, so the trim's
6810-token result already fits comfortably here regardless of the
5120-vs-6810 gap; the previously-failing test now passes for real (the
model generates and returns "OK", not a fallback/error path), independently
verified via NODESPACE_PROMPT_DUMP + the real tokenizer, not assumed. The
per-turn silent-failure fix in the same PR is what makes any machine whose
window IS smaller than its own prompt fail visibly instead of silently,
regardless of the exact number on that machine.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mstomar125

Copy link
Copy Markdown
Contributor Author

APPROVE

Adversarial review (general-purpose subagent, high effort) ran against the real diff — no blockers. Two non-blocking items and two nits, both addressed in the follow-up commit:

  • Restored a dropped disambiguation clause in route_clarify's description (semantic-narrowing nit).
  • Documented, in-code, why a synthesized failure message is deliberately fed back into the model's own future history unfiltered (duplicate-write guard unaffected since completed_writes is empty; growth bounded by the existing maybe_summarize_history).
  • Reconciled the commit-message token math against the issue's reported 5120-token window vs. this machine's actual (larger) ADR-062-computed window — the previously-failing test now passes for real, independently verified via NODESPACE_PROMPT_DUMP + the real tokenizer.

Everything the reviewer verified independently (turn_error threading, append_assistant_message fallback, golden-file diffs, the clippy fix's correctness, cancellation-path regression coverage, scope) came back clean. Pushed through the full pre-push gate (scripts/test-gate.ts) twice, clean both times, including the target ADR-048 test.

Merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ai-chat inference turns silently fail when system prompt exceeds the loaded model's context window

1 participant