ai-chat inference turns silently fail when system prompt exceeds the loaded model's context window #2167

Overview

The pre-push gate's Tauri-seam integration suite (cargo test -p nodespace-app, ADR-048) is currently failing deterministically on this machine, blocking every push to main regardless of what the pushed diff touches. The failing test looks like a 300-second hang, but it is not — the daemon fails the underlying inference turn almost immediately and silently swallows the failure, so the test's polling loop has no way to detect it and burns the full timeout before panicking with a misleading message.

Problem Statement

Observed test failure:

test ai_chat_send_reaches_idle_with_no_stuck_processing_state ... FAILED
thread panicked at packages/desktop-app/src-tauri/tests/ai_chat_send_to_idle_test.rs:98:13:
ai-chat turn did not reach idle-with-new-reply within 300s; last status="idle", assistant_count=0

This reads as a hang: status="idle" (not stuck "processing") but assistant_count=0 (no reply ever arrived), so the test's polling loop (status == "idle" && assistant_count > prior_assistant_count) never satisfies and eats the full 300s timeout.

Actual root cause, found by re-running with E2E_VERBOSE=1 (which surfaces the spawned daemon's own stdout/stderr, normally discarded): the daemon logs this about 19 seconds into the run, long before the test's 300s deadline:

WARN nodespace_daemon::services::local_agent_service: inference turn failed
node_id="<uuid>" error=inference engine error: Context window exceeded:
Prompt uses 6032 tokens but context window is 5120

So the inference turn fails fast and deterministically — the system prompt (base instructions + tool definitions) plus the turn's message no longer fits inside the model's context window. The daemon then resets the node's ai-chat status back to "idle" without appending an assistant message and without surfacing the failure anywhere the caller can observe — so poll_until_idle_with_new_assistant_reply in the test, and (by the same mechanism) any real client polling this same node shape, sees only "idle, no new message" forever. The 300s test timeout is not the bug; it is a symptom of a turn failure with no visible signal.

This is not resource contention. Reproduced 3 times total: twice via the real pre-push gate (scripts/test-gate.ts, using its exact NODESPACED_TEST_BIN=target/debug/nodespaced + --test-threads=1 invocation) and once via a fully isolated manual run with the same binary and flags, load average ~1.3–1.35, and zero other cargo/bun/nodespaced processes running at the time. All three runs failed identically with the same context-window-exceeded log line. A prior hypothesis (cross-worktree build/test contention, based on a similar-looking failure in an unrelated Vitest suite on a different worktree around the same time) was checked and ruled out for this specific test by the isolated, quiet-system rerun.

Evidence

• Model: native llama.cpp path, gemma-4-e4b-q4km (locked per ADR-056).
• Context window: computed at model-load time, budgeted against total physical RAM per ADR-062 — deterministic per machine+model, 5120 tokens on this machine.
• Failing prompt size for this specific turn: 6032 tokens — 912 tokens over budget, for a turn whose only user content is "Reply with exactly one word: OK" (packages/desktop-app/src-tauri/tests/ai_chat_send_to_idle_test.rs). The overage is coming from the system prompt / tool-schema surface, not the user message.
• ADR-062 already establishes the load-time case: "A computed window too small to hold the agent's system prompt plus a working turn is a load-time error naming total RAM, weight size, and the shortfall. Silently clamping to a small floor produces a session in which every tool-using turn fails with an opaque context-window error." That reasoning appears to hold here too, but this failure happens after a successful load (EnsureModelReady completes cleanly), during a specific turn — a case ADR-062's text does not explicitly cover, and the current behavior (silent reset to idle, no error surfaced) is the exact "opaque context-window error" outcome ADR-062 says is worse than an actionable one.
• Likely contributor: this session landed several commits shortly before this failure surfaced that grow the tool/schema content included in every turn's system prompt — most plausibly Register a Stage-2-callable route_clarify tool #2161 (Register a Stage-2-callable route_clarify tool) and Expose friendly_name in create_schema/update_schema tool schemas + skill guidance #2164 (Expose friendly_name in create_schema/update_schema tool schemas + skill guidance). Neither has been confirmed as the specific commit that tipped total prompt size from under 5120 to 6032 tokens — that measurement is investigation work, not done here.

Impact

Blocks scripts/test-gate.ts's cargo test -p nodespace-app (Tauri-seam integration tests, ADR-048) step — and therefore git push — for every push to nodespace-core on any machine where this model is already provisioned under ~/.nodespace/models/, independent of what the pushed diff touches. Not flaky: it is deterministic once the model is loaded at this machine's ADR-062-computed window.

Acceptance Criteria

• Root cause of the prompt-size growth (which commit(s)/schema additions pushed total tokens from under 5120 to 6032) is identified and measured, not assumed
• ai_chat_send_reaches_idle_with_no_stuck_processing_state passes reliably again
• A turn whose prompt exceeds the loaded context window fails visibly (surfaced to the caller / recorded on the node) rather than silently resetting to "idle" with no new message — regardless of which specific fix (trimming prompt content, raising the window, chunking/summarizing, or something else) is chosen
• bun run test:all and the full pre-push gate (scripts/test-gate.ts) pass cleanly on a clean checkout

Technical Specifications

Reference Files

• Test: packages/desktop-app/src-tauri/tests/ai_chat_send_to_idle_test.rs — the failing integration test; its doc comment explains why it exercises the real native model rather than a stub
• Daemon inference path: nodespace_daemon::services::local_agent_service (source of the WARN ... inference turn failed log line) — packages/daemon/src/services/local_agent_service.rs
• Context window sizing: implements ADR-062's RAM-based budget; governs the 5120-token figure for this machine+model
• Verbose daemon output: E2E_VERBOSE=1 env var (test-support harness) surfaces the spawned daemon's stdout/stderr, otherwise discarded — needed to see the log line above; packages/desktop-app/src-tauri/test-support/src/lib.rs

Related Documentation

• ADR-056 — locks the native path to gemma-4-e4b-q4km
• ADR-062 — establishes deterministic, RAM-based context-window sizing and the "refuse loudly, don't clamp silently" principle this issue's silent-turn-failure behavior appears to violate at the per-turn (not per-load) level

Related Issues

• Register a Stage-2-callable route_clarify tool #2161 (likely contributor — grows tool schema surface)
• Expose friendly_name in create_schema/update_schema tool schemas + skill guidance #2164 (likely contributor — grows tool schema surface)