Upgrade vendored llama.cpp so local models route tools reliably (gates the skill-routing redesign; supersedes #1329 verdict) #1355

Overview

NodeSpace's local inference uses llama-cpp-2 0.1.146, whose vendored llama.cpp C++ source lags well behind upstream. Empirical testing (build 8660 via llama-server, same GGUFs, native chat templates) showed that tool-call failures previously attributed to model capability are caused by this old vendored engine.

This issue upgrades the vendored llama.cpp and verifies the fix with pinned regression fixtures. It is the gating prerequisite for the skill-mediated tool-routing redesign (ADR-036) — that design was validated on build 8660, not on our current binding.

Evidence (NodeSpace-independent, current engine)

• Gemma 4 12B: inside NodeSpace = "near-zero tool use, fabricates" (the basis for A/B test Gemma 4 12B (Q4_K_M) vs E4B for the local ai-chat agent #1329 closing "keep E4B"). On current llama.cpp = one clean, valid create_schema call.
• Ministral 8B: inside NodeSpace = malformed JSON {"name":"amount_due","number"} (dropped type key), looped 5×. On current llama.cpp = flawless nested tool call.

This invalidates #1329's negative verdict — it measured the broken binding, not model capability.

Proposed Solution

• Bump llama-cpp-2 / its vendored llama.cpp to a current build (target parity with the build-8660 behavior verified in testing).
• Re-verify tool calling across the shipped models.

Acceptance Criteria

• Vendored llama.cpp upgraded; cargo build green with metal feature.
• Regression fixtures pinned: the two historical failures (Ministral 8B dropped-type malformed create_schema; Gemma 4 12B near-zero tool use) are encoded as tests that now PASS.
• [TOOL_CALLS] Mistral parser still works — the custom parser ADR-030 deliberately preserved is verified against the upgraded engine (its own acceptance criterion, since the bump risks breaking it).
• Tool-calling smoke test green for: Gemma 4 E4B/12B, Ministral 3B/8B, Qwen3-8B, Mistral Small 3.2.
• No regression in embedding generation (nomic-embed path).
• Code passes bun run quality:fix; tests pass.

Technical Specifications

Reference Files

• packages/nlp-engine/Cargo.toml — llama-cpp-2 pin (>=0.1.146, features metal).
• packages/nlp-engine/src/chat/mod.rs — sampler chain, OpenAIChatTemplateParams (json_schema/grammar currently None), [TOOL_CALLS] handling.
• packages/agent/src/local_agent/ — tool-call parsing for Gemma 4 template (thought channel, <turn|> EOG).

Notes

• Build 8660 also exposes --json-schema / --grammar constrained decoding (currently plumbed but unwired in OpenAIChatTemplateParams). Wiring grammar-constrained tool-call decoding is a possible follow-up, out of scope here.

Related Issues

• A/B test Gemma 4 12B (Q4_K_M) vs E4B for the local ai-chat agent #1329 (verdict superseded by this upgrade + ADR-036)
• ADR-036 (skill-mediated routing — gated behind this)
• Model-capability guardrails + larger-model prerequisites (anti-fabrication, KV-cache quant, per-model n_ctx, Gemma-12B tool parsing) #1332 (model-capability guardrails / larger-model prerequisites)

Non-Goals

• The routing redesign itself (separate issues, gated on this).
• Grammar-constrained decoding wiring (possible follow-up).