Repository navigation
Trim #2161/#2164 tool-schema bloat; surface turn-level inference failures - #2168
Merged
Merged
Conversation
…ures Measured the actual assembled prompt for the failing ai-chat scenario via NODESPACE_PROMPT_DUMP + the real tokenizer: on this machine the turn's full-tool-surface prompt is 6932 tokens (713 resident prose + 6202 across all 13 fail-open tools + 17 trailer), with create_schema (1341) and update_schema (1034) — the two tools #2164 touched — together accounting for over a third of the tool-schema budget. declare_write_tool_fields contributes zero extra tokens for this scenario (no skill candidate clears the score gate, so field declarations never fire). Trims applied within the tool-schema/skill-guidance channels (ADR-064): - create_schema's per-field name/friendlyName/description descriptions shortened to match update_schema's already-tighter phrasing instead of carrying two different-length explanations of the same argument shape. - update_schema's rename_fields description tightened (identical facts, half the words) — it was restating, almost verbatim, what the RENAME_VS_RELABEL skill rule also says. - RENAME_VS_RELABEL (skill_rules.rs) cut back to the procedural judgment call (which shape the user's intent maps to); the mechanical "from"/"to"/ "friendlyName" explanation now lives only in the tool schema, its correct channel. Regenerated packages/skill/SKILL.md from the trimmed rule. - route_clarify's (#2161) description tightened slightly. - Fixed a pre-existing `cargo clippy --all-targets -D clippy::unused-async` failure in the same file: exec_route_clarify was declared async with no .await in its body (from #2161); made it a plain sync fn. Net: 6932 -> 6810 tokens for the measured scenario (122 saved, confirmed via the real tokenizer, not estimated). The prompt_assembly_snapshot goldens were regenerated to pin the new tool-schema/skill-guidance text. Also fixes the silent-failure half of this issue: local_agent_service.rs's run_ai_chat_turn logged a WARN and reset the node straight to "idle" on any inference failure (including ContextOverflow) with no new assistant message and no way for a polling caller or the frontend to distinguish a failed turn from one still in flight. It now appends an assistant-role message naming the failure and reaching idle through the same path a normal reply uses, matching ADR-062's "refuse loudly, don't clamp silently" principle applied at the per-turn level. Added failed_inference_turn_surfaces_a_visible_error_not_silent_idle as a regression guard. Closes #2167. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… history tradeoff Adversarial review of PR #2168 found no blockers. Two non-blocking items addressed here: - route_clarify's description had dropped "and nothing already said picks one" when trimmed — restored (a few tokens) since without it the text reads as license to clarify on any multi-match search result even when other conversational context already disambiguates. - Documented, at the append_assistant_message call for a failed turn, why feeding the synthesized failure text back into the model's own future history as an ordinary assistant message is accepted rather than filtered/tagged: it can't trip the duplicate-write guard (empty completed_writes), unbounded growth is bounded by the existing maybe_summarize_history mechanism, and a model that sees its own stated failure has exactly the context needed to avoid blindly repeating it. Also: the review noted this PR's first commit message states a net 6932 -> 6810 token result against the issue's reported 5120-token window without reconciling that 6810 > 5120 — worth stating plainly here since that commit's own text doesn't: the 5120 figure is this issue's reported window on ITS reporting machine. On this machine, ADR-062's `fit_n_ctx_to_budget` (packages/nlp-engine/src/chat/mod.rs) grants the full configured 32768-token window given its total RAM, so the trim's 6810-token result already fits comfortably here regardless of the 5120-vs-6810 gap; the previously-failing test now passes for real (the model generates and returns "OK", not a fallback/error path), independently verified via NODESPACE_PROMPT_DUMP + the real tokenizer, not assumed. The per-turn silent-failure fix in the same PR is what makes any machine whose window IS smaller than its own prompt fail visibly instead of silently, regardless of the exact number on that machine. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
Author
|
APPROVE Adversarial review (general-purpose subagent, high effort) ran against the real diff — no blockers. Two non-blocking items and two nits, both addressed in the follow-up commit:
Everything the reviewer verified independently (turn_error threading, append_assistant_message fallback, golden-file diffs, the clippy fix's correctness, cancellation-path regression coverage, scope) came back clean. Pushed through the full pre-push gate ( Merging. |
This was referenced Aug 19, 2026
7 tasks
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #2167.
The
ai_chat_send_reaches_idle_with_no_stuck_processing_stateintegration test's underlying scenario was reported failing withPrompt uses 6032 tokens but context window is 5120— the assembled system prompt for a fail-open (no skill matched) turn exceeding this machine's ADR-062-computed context window. This PR does the two things the issue's acceptance criteria call for.1. Measured, not assumed
Reproduced the exact failing-test scenario and captured the real assembled prompt via
NODESPACE_PROMPT_DUMP(the existing dev tool for this), then tokenized every tool-schema segment with the real locked model's tokenizer (not char-count estimates). On this machine the turn is a fail-open turn (no skill candidate clears the routing score gate for "Reply with exactly one word: OK"), so it gets the full unscoped tool surface: 6932 tokens = 713 (resident prose) + 6202 (all 13 tool schemas) + 17 (trailer).create_schema(1341 tok) andupdate_schema(1034 tok) — the two tools #2164 (friendly_name exposure) touched — together account for over a third of the tool-schema budget.declare_write_tool_fields(#2120/#2148/#2147) contributes zero extra tokens for this scenario, confirmed by direct inspection: no skill candidate clears the score gate, so no field declarations are generated.(Note: on this dev machine,
fit_n_ctx_to_budgetgrants the full configured 32768-token window given its RAM, so this specific run doesn't itself hit the 5120-token ceiling the issue reports for a more memory-constrained machine — the token-size finding and trim above are still real and machine-independent; the fix in part 2 below is the actual safety net for any machine where the window IS smaller.)2. Trimmed, within the correct ADR-064 channel
create_schema's per-fieldname/friendlyName/descriptionschema descriptions shortened to matchupdate_schema's already-tighter phrasing for the same argument shape (previously two different-length explanations of the same thing).update_schema.rename_fields's description tightened — same facts, roughly half the words. It was restating almost verbatim what theRENAME_VS_RELABELskill rule already says.RENAME_VS_RELABEL(skill guidance) cut back to only the procedural judgment call (whichfrom/toshape a user's intent maps to); the mechanical explanation now lives solely in the tool schema, per ADR-064 rule 1. Regeneratedpackages/skill/SKILL.mdfrom the trimmed rule viagen_skill_md.rs --write.route_clarify's (Register a Stage-2-callable route_clarify tool #2161) description tightened slightly.cargo clippy --all-targets -D clippy::unused-asyncfailure in the same file (exec_route_clarifydeclaredasyncwith no.awaitin its body, from Register a Stage-2-callable route_clarify tool #2161) — this was blocking a clean clippy run independent of this issue, in code this PR already touches.Net measured result for the scenario above: 6932 → 6810 tokens (122 saved, confirmed via the real tokenizer). Regenerated the
prompt_assembly_snapshotgoldens (#2119/#2120) to pin the new text — diffs reviewed, each one is exactly the intentional trim above.3. Silent-failure fix (the issue's 3rd acceptance criterion)
local_agent_service.rs::run_ai_chat_turnpreviously logged aWARNand reset the node straight to"idle"on ANY inference failure (context overflow included), with no new assistant message — indistinguishable, to a caller pollingget_node(or the frontend, which only rendersuser/assistantmessages), from a turn still in flight. It now appends an assistant-role message naming the failure and reachesidlethrough the same path a normal reply uses (append_assistant_message), matching ADR-062's "refuse loudly, don't clamp silently" principle applied at the per-turn level, not just at model load. A genuine cancellation still resets silently (that's an intentional user action, not a failure).Added
failed_inference_turn_surfaces_a_visible_error_not_silent_idleas a regression guard (aFailingEnginestub returningInferenceError::ContextOverflow, asserting the node reaches idle WITH a new assistant message naming the failure).Verification
cargo test -p nodespace-agent --lib: 522 passedcargo test -p nodespace-daemon --lib: 153 passed (includes the new regression test)cargo test -p nodespace-agent --test prompt_assembly_snapshot: 5/5 passed against regenerated goldenscargo test -p nodespace-app --test ai_chat_send_to_idle_test ai_chat_send_reaches_idle_with_no_stuck_processing_state -- --test-threads=1: passed in isolation (no concurrent worktree builds)cargo clippy --all-targets -- -D warnings -D clippy::unused-async: clean across the whole workspacecargo fmt --checkon every file this PR touches: cleanbun run test:all: full pass (frontend, skill, Rust workspace)scripts/test-gate.ts(the real pre-push gate): ran clean end-to-end viagit push, including the target ADR-048 suiteTest plan
ai_chat_send_reaches_idle_with_no_stuck_processing_statepasses reliablybun run test:alland the full pre-push gate pass cleanly on a clean checkout🤖 Generated with Claude Code