Shared eval runner with a preflight gate: stop scoring scenarios when the environment is broken, and make a new eval a fixture instead of a copied harness

[malibio]

Closes #1759

[malibio]

Code Review: PR #1778 — Shared eval runner with a preflight gate

Review Type: Initial Review (single commit de6413d0, 18 files, +1297/-779)

---

Requirements Check

#	Acceptance Criterion	Status
1	Shared runner exists; both evals ported with no behavior change	✅ Verified byte-for-byte against the deleted originals
2	Unreachable daemon / unloaded model / too-small context aborts with clear error + distinct exit code, never partial results	⚠️ Mostly — gate is correct at startup, but a mid-run daemon death still produces partial results (see 🔴 #1)
3	Adding an eval requires only a fixture module	✅ EvalFixture interface + 10-line entry wrapper; no plumbing
4	Both evals compare against a committed baseline by default	⚠️ Deliberately reinterpreted — see "Deviation 1" below
5	Results record model, date, host memory, granted n_ctx	✅ Provenance adds commit + dirty flag beyond the ask
6	test:all runs the scoring test	✅ test:scripts in both test:all and test:all:db; 21 tests, 57ms
7	Eval setup documented in nodespace-docs	✅ development/agent-eval.md; scripts/aichat-1329-trial.md deleted

---

Assessment of the Deliberate Deviations

Deviation 1 — no committed baseline (AC #4). Sound; endorse. Committing results-e4b.json (11/12) would bake in a number that #1760 exists to retract, #1764 owns re-recording, and #1775 will move again. The issue's own framing — "a number that outlived the conditions that produced it" — argues against committing this particular figure. The PR substitutes something stronger: mandatory Provenance in every results file makes a number self-dating, so it cannot outlive its conditions. That addresses the actual failure mode; "compare by default" was the mechanism, not the goal.

The residual gap is real but small: comparison is now opt-in, so a run that forgets --baseline silently reports no regressions. scripts/eval/runner.ts:358 — mitigable in a follow-up by defaulting --baseline to a conventional path when it exists (see 🟢 #7).

Deviation 2 — the 4th preflight check + proto/daemon/CLI change. Belongs here; endorse. The check was added because this branch's own eval runs wrote schemas and chat nodes into ~/.nodespace. A gate whose entire purpose is "refuse to run when the environment cannot produce a valid result" that permits corrupting live user data would be incomplete on its own terms. It is not scope creep — it is the same requirement applied to a failure mode discovered during implementation.

The proto/daemon/CLI change is likewise load-bearing rather than opportunistic: AC #2 and AC #5 both require granted_n_ctx, and that value existed only in-process. There was no way to satisfy the issue without exposing it. Implementation is correct — reporting the catalog id from active_model_id rather than the resolved GGUF path (local_agent_service.rs:726-733) is the right call, since no path substring matches gemma-4-e4b-q4km.

---

Code Review Findings

🔴 Critical

1. 📁 scripts/eval/runner.ts:69-96 — a mid-run daemon death still produces the exact false-pass the issue was written about

This is the highest-value item. runTurn swallows a non-zero exit and returns toolsCalled: []. newChat throws EnvironmentError, but only fires between groups. The agent-matrix fixture has three groups, so a daemon that dies during group 2 or 3 produces:

runTurn → exitCode != 0 → { toolsCalled: [], reply: "(send failed: ...)" }
→ assertExpectation({kind:"noTools"}, []) → { passed: true }

Scenarios 1 and 2 are noTools — they are scored in groups 1 and 2, before any newChat would catch the death. The routing fixture has 15 single-scenario groups, so newChat does fire between each — but the daemon can die after newChat succeeds and during the send; clarify scenarios fail correctly there since isClarification("(send failed: ...)") returns false. The noTools matrix scenarios are the live hazard.

The main agent hit precisely this during testing — a run reported "9/12 scenarios failed" from a dead daemon, meaning 3 scored as passes from a corpse.

The preflight gate proves the environment was good at t=0. It does not prove it stayed good, and an eval run is 8+ minutes. The principle at stake is the issue's own: an eval that can silently score a broken environment is worse than no eval, because its output gets cited.

Suggested fix — a consecutive-failure abort in the run loop:

let consecutiveFailures = 0;
// inside the scenario loop, after `const scored = runTurn(...)`:
if (scored.reply.startsWith("(send failed:")) {
  if (++consecutiveFailures >= 2) {
    throw new EnvironmentError(
      `${consecutiveFailures} consecutive turns failed to send — the daemon went away mid-run.`,
      `Partial scores from an aborted run are not comparable to a complete one, so none were written.`,
    );
  }
} else {
  consecutiveFailures = 0;
}

The existing catch at runner.ts:309-312 already routes EnvironmentError to abortOnEnvironment, so this needs no new plumbing. A threshold of 2 tolerates a single flaky turn while catching a systemic death within one extra turn instead of a whole run.

---

🟡 Important

2. 📁 scripts/eval/preflight.ts:158-173 — readServedDatabasePath fails open, defeating the destructive-risk check exactly when it matters

Both error paths return "", and preflight at line 193 guards on status.databasePath && — so an empty string skips the check entirely. The failure modes that produce "" are: the CLI has no database list subcommand (older build), the RPC fails, or the JSON shape changed. In each case the daemon is still serving something, and the eval proceeds to write into it.

For the other three checks, failing open is merely a missed detection. For this one it is irreversible damage to live user data — the specific harm that motivated adding the check. A guard against an unrecoverable outcome should fail closed.

Recommend throwing EnvironmentError when the path cannot be determined, with a remedy explaining how to confirm isolation (grep served_db_path <daemon log>).

Secondary: the check only catches $HOME/.nodespace/. A daemon serving any other real database passes. Inverting the check to require the served path be under the configured test home would be strictly safer.

3. 📁 scripts/eval/preflight.ts:223 — status.modelId.includes(env.model) admits false positives on model families

• NS_MODEL=gemma-4-e4b matches loaded gemma-4-e4b-q4km ✅ intended
• NS_MODEL=gemma-4-e4b also matches loaded gemma-4-e4b-q8 ❌ different quantization, different scores

The check exists so scores are not filed against a model that did not produce them — quantization changes tool-calling behavior materially. Since local_agent_service.rs:726-733 now returns active_model_id, exact equality should hold in the normal path; containment is only needed for the fallback where active_model_id is None. Suggest equality first, path-basename fallback, warn when only the fallback matched.

4. 📁 packages/daemon/src/services/local_agent_service.rs:718-721 — get_status can now block a tokio worker for the length of a generation

service.model_spec() → engine().model_info() → LlamaChatEngine::model_info() (chat/mod.rs:1120) takes a std::sync::Mutex. generate_blocking (chat/mod.rs:343) holds that same mutex for the entire duration of inference. So get_status — previously cheap — now blocks on an inference that can run 60–180s, on a tokio worker thread, while holding service.read().

Mitigating: local_agent_status is currently unused by the frontend (only the new CLI path calls it), and service.read() is a read lock. But this is a latent trap: the moment a status poller is added to the UI, it will hang during streaming.

Lock ordering is safe — every site touching service and active_model_id (lines 161, 194–210, 215–221, 720–726, 1119, 1251) acquires service first and releases before active_model_id. No deadlock.

Suggest caching the spec at load time alongside active_model_id (it only changes on engine swap, and replace_engine is the single choke point), or documenting the blocking behavior.

5. 📁 packages/daemon/src/services/local_agent_service.rs:720 — unwrap_or(None) collapses "engine error" into "no model loaded"

An InferenceError and a genuinely absent model are indistinguishable to the caller; preflight will suggest model load ... for what may be an engine fault. A tracing::warn! on the error arm preserves degrade-don't-fail while leaving a trail.

---

🟢 Suggestions

6. 📁 scripts/eval/env.ts:11 — stale doc path. References agent-evals.md; the file is agent-eval.md (singular). preflight.ts:121 has it right.

7. 📁 scripts/eval/runner.ts:358 — closing the AC #4 gap cheaply. Once #1764 records the honest baseline, default baselinePath to a conventional location when the file exists, keeping --baseline as an override. Worth a note on #1764.

8. 📁 scripts/eval/types.ts:47 — the [key: string]: unknown index signature weakens typo detection, but the alternative is worse. Making EvalFixture<S extends Scenario = Scenario> generic would remove both the index signature and every cast. Right shape, but a refactor of a brand-new abstraction with two consumers — reasonable to defer until a third eval exists.

9. 📁 packages/agent/src/agent_guidance.rs:318-340 — the read_dir guard is well-built; one gap. Anti-vacuity assertions are excellent and verified non-vacuous. Gap: a non-fixture .ts file added to fixtures/ (a shared helper) would have no prompt: literals and fail the build with a misleading parser-drift message. Consider filtering on presence of a prompt: literal and asserting the set is non-empty.

10. 📁 scripts/eval/preflight.ts:67,78 — constants verified. AGENT_SYSTEM_PROMPT_TOKENS = 6600 matches N_CTX_MINIMUM's doc comment (chat/mod.rs:1352-1358) verbatim. 6600 + 4096 = 10,696 sits below N_CTX_MINIMUM = 16_384, so the gate never rejects a window the engine itself accepts — correctly a floor, not a duplicate. Nit: nothing enforces the "kept in step with" claim.

11. 📁 package.json:23 — test:scripts scope is exact and the bun test exception is legitimate. find scripts -name "*.test.ts" returns exactly one file. Forward risk: a future scripts/*.test.ts needing DOM would be silently misrouted — worth a line in CLAUDE.md noting scripts/ tests must stay DOM-free.

12. Nit: 📁 scripts/eval/env.ts:29,39 — timeoutMs and log are dead in the runner. Both read into EvalEnv but never used; aichat.ts reads them from process.env directly. Per the "delete unused code" standard, either drop them or comment that they document the child process's contract.

13. Nit: 📁 scripts/eval/runner.ts:220-222,229 — usage errors exit EXIT_FAILED (1), same as a scenario failure. Argv parsing is otherwise robust (flag-before-label, flag-at-end, missing-value all tested).

---

Verification performed

• Behavior preservation — verified. Diffed both fixtures against git show origin/main:scripts/aichat-matrix.ts and :scripts/routing-eval.ts. All 12 matrix scenarios and all 15 routing fixtures byte-identical (prompts, expectations, flags, priorTurns, group boundaries). assertExpectation and assertFixture moved verbatim including failure strings. All routing helpers transcribed intact. The old assertionTurns slice is preserved by the runner passing [scored] to both score and extra.
• bun run test:scripts → 21 pass / 0 fail in 57ms; bun test scripts/ matches exactly one file.
• cargo test -p nodespace-agent guidance_is_not_contaminated_by_eval_prompts → passes against the new read_dir enumeration.

---

Recommendation

REQUEST CHANGES

This is a strong, well-reasoned PR and a clear net improvement: it eliminates five duplicated concerns, makes adding an eval a fixture file, ports both evals with verified byte-for-byte scoring fidelity, and closes the silent-corruption hole the issue was written about. Both deviations from the issue text are well-argued and endorsed.

Two items should land before merge, both in the load-bearing preflight/runner path:

• 🔴 Set up Tauri + Svelte project structure #1 — mid-run daemon death still scores noTools scenarios as passes. This is the precise failure mode the issue exists to eliminate, and the fix is ~10 lines reusing plumbing already written.
• 🟡 Create design system foundation #2 — readServedDatabasePath failing open. Every other check failing open costs a missed detection; this one costs irreversible writes to live user data.

🟡 #3, #4, and #5 are worth addressing but would not block on their own — #4 is latent rather than active, since nothing polls that RPC today. The 🟢 items are polish and follow-up material.

No architectural regression, and no project standard violated — no any, no lint suppression, no backward-compat scaffolding, no #[allow(dead_code)], Bun-only, docs correctly in nodespace-docs with no issue numbers in doc content.

[malibio]

Review findings addressed

Commits 61ba6d0d (checkpoint) + 4ed1af7d.

✅ Addressed (9)

#	Finding	What changed
🔴 1	Mid-run daemon death scored noTools as passes	TurnRecord.sendFailed distinguishes "never reached the model" from "called no tools"; two consecutive failed sends abort as an environment error
🟡 2	readServedDatabasePath failed open	Now fails closed — throws on non-zero exit, unparseable JSON, or empty database list
🟡 3	Substring model match accepted wrong quantization	Exact match first, filename fallback for the resolved-path case, warning when only the fallback matched
🟡 5	unwrap_or(None) hid engine errors	tracing::warn! on the error arm; degrade-don't-fail behavior unchanged
🟢 6	Stale doc path	agent-evals.md → agent-eval.md
🟢 9	Guard would fail confusingly on a non-fixture helper	Fixtures now selected by whether they declare scenario prompts, not filename
🟢 11	test:scripts exception could widen	CLAUDE.md records that tests under scripts/ must stay DOM-free
🟢 12	Dead timeoutMs/log fields	Documented as the contract aichat.ts consumes from the environment
🟢 13	Usage errors shared exit 1	Usage now exits 64

Also fixed the abort footer, which claimed "no scenarios were run" when a mid-run abort had already scored some.

Verification of the two blocking items

🔴 #1 — reproduced the original bug, then confirmed the fix. Live daemon, killed mid-group (inside the 5-scenario chain, where newChat never fires):

• Before: 9/12 scenarios failed, exit 1, results file written — 3 dead-daemon turns scored as passes.
• After: exit 2, 10 scenario(s) had been scored; they are NOT written, no results file.

🟡 #2 — confirmed the destructive check still wins. With a model loaded and the daemon serving the real database via the NODESPACED_DB_PATH registry-override trap, the DB refusal fires ahead of the model check.

Also re-verified the contamination guard after the #9 change: a helper file with no prompt: literals is skipped, and a planted example still fails the build (checked with a scenario-8c prompt pasted into SCHEMA_CREATION_RULES) — so the guard is not vacuous.

⏭️ Deferred (4), with reasons

• 🟡 Enhanced BaseNode with Hierarchical Circle Indicators + TextNode Implementation #4 — get_status can block a tokio worker for the length of a generation. Real, but latent: nothing polls that RPC today (only the new CLI path, which runs before generation). The fix is a daemon-internal refactor — cache the spec at the replace_engine choke point — which is beyond this PR's scope. Filed as GetStatus blocks a tokio worker for the length of a generation #1779.
• 🟢 Svelte frontend renders with placeholder panels #7 — default --baseline path. Depends on Make the agent eval diagnostic before recording a baseline; revisit ADR-056's Scenario 5 claim #1764 recording the honest baseline first; wiring it now would default to a file that does not exist.
• 🟢 Development server works with hot reload #8 — generic EvalFixture<S extends Scenario>. Agreed it is the right shape, but it is a refactor of a brand-new abstraction with two consumers — the reviewer's own YAGNI call. Worth doing when a third eval arrives.
• 🟢 No compilation errors or warnings #10 — enforce that AGENT_SYSTEM_PROMPT_TOKENS tracks N_CTX_MINIMUM. Both sites carry cross-referencing comments, and the value is deliberately a floor rather than an exact accounting, so drift is tolerable by construction.

Tests

bun run test:all → exit 0. Frontend 4096/4096 across 169 files (unchanged from baseline), skill 20/20, Rust suites green. test:scripts 21/21. Pre-push gate (incl. e2e) passed.

[malibio]

Code Review: PR #1778 — Re-Review

Review Type: Re-Review
Delta reviewed: de6413d0..HEAD (61ba6d0d, 4ed1af7d) — 10 files, +233/−68
Scope: the two new commits only. Behavior-preservation, lock ordering, and constant verification from the original review carry forward unchanged (the delta does not touch those areas).

---

Previous Review Summary

	Count
Findings in original review	13 (1 🔴, 4 🟡, 8 🟢)
Addressed	9 — #1, #2, #3, #5, #6, #9, #11, #12, #13
Deferred with reasons	4 — #4 (filed as #1779), #7, #8, #10
New issues found in delta	1 🟡 residual, 2 🟢 nits

All four deferrals are legitimate. #4 is latent (nothing polls the RPC) and its fix is a daemon-internal refactor at the replace_engine choke point — correctly out of scope and tracked. #7 genuinely blocks on a baseline that does not yet exist; defaulting to a nonexistent file would be worse than opt-in. #8 was my own YAGNI call, and #10 is tolerable by construction since the constant is a floor rather than an exact accounting.

---

Requirements Check — issue #1759

#	Acceptance Criterion	Status
1	Shared runner; both evals ported with no behavior change	✅ Re-verified: delta's fixture edits are formatting-only
2	Unreachable daemon / unloaded model / too-small context aborts, never partial results	⚠️ Substantially closed, one hole remains (see 🟡 A)
3	Adding an eval requires only a fixture module	✅ Unchanged
4	Compare against a committed baseline by default	⚠️ Deviation endorsed in original review; unchanged
5	Results record model, date, host memory, granted n_ctx	✅ Unchanged
6	test:all runs the scoring test	✅ 21/21 in 65 ms, re-run
7	Eval setup documented in nodespace-docs	✅ Path reference corrected

The specific case the original review named — a noTools matrix scenario scoring a corpse as a pass — is genuinely fixed. A structurally identical case survives on the priorTurns path.

---

Findings

🟡 Important

A. 📁 scripts/eval/runner.ts:296-317 — the abort covers scored turns but not priorTurns

runTurn is called on two paths; the new sendFailed check guards only one:

for (const prior of scenario.priorTurns ?? []) {
  priorTurns.push(runTurn(env, chatId, prior));   // ← no sendFailed check
}
const scored = runTurn(env, chatId, scenario.prompt);
if (scored.sendFailed) { ... }                     // ← guarded

Two routing fixtures use priorTurns, both loadBearing: true:

• instance-not-schema-invoice — 1 prior turn, the only fixture testing "existing type → Node Creation, not Schema Creation"
• clarification-then-fallthrough — 2 prior turns, the only fixture testing the clarification contract

The uncaught case: a prior turn fails but the scored turn succeeds. The counter never increments, so the scenario runs against a chat that never received its context — and clarification-then-fallthrough expects {kind: "search"} from a model that was never clarified to. That scores as a legitimate routing verdict when it is an environment artifact. A silent failure is less dangerous than a silent pass, but it is still a number filed under "the model got routing wrong" when the truth is "the fixture never ran as designed."

Also: since priorTurns never touch the counter, a daemon that dies during a 2-prior-turn setup and stays dead is only caught two scenarios later, after burning three failed sends.

Fix reuses what is already written:

const record = (t: TurnRecord) => {
  if (t.sendFailed) {
    if (++consecutiveSendFailures >= MAX_CONSECUTIVE_SEND_FAILURES) throw new EnvironmentError(...);
  } else {
    consecutiveSendFailures = 0;
  }
  return t;
};
priorTurns.push(record(runTurn(env, chatId, prior)));
const scored = record(runTurn(env, chatId, scenario.prompt));

Engineering principle: a guard placed on one of two structurally identical call sites is a guard with a documented bypass. The comment on MAX_CONSECUTIVE_SEND_FAILURES — "if the daemon dies mid-run every subsequent turn calls no tools" — applies verbatim to prior turns, so the omission reads as oversight rather than decision.

Not a merge blocker on its own. The 🔴 that motivated REQUEST CHANGES is genuinely fixed; this is a smaller, differently-shaped version on a two-fixture path.

---

Verification of the items probed hard

🔴 #1 — counter semantics, single-failure behavior, threshold. All correct.

• Reset: consecutiveSendFailures = 0 on every non-failed send — measures a run, not a total. ✅
• Single failure still scores normally: 1 >= 2 is false, control falls through to fixture.score(...) exactly as before. This was the property most at risk from an over-eager fix, and it survived. ✅
• sendFailed on every failing path: runTurn has exactly one non-success return (runner.ts:80-89) and it sets the flag. No third path. newChat throws directly — different mechanism, already correct. ✅
• Threshold of 2 vs group boundaries: the group structure makes it stronger. newChat fires at each boundary and throws immediately, so the counter only has to survive within a group. Worst case (group 3, 8a–8e): death at 8a aborts at 8b, one wasted turn. ✅
• Abort path traced: throw at :307 → catch at :344-347 → abortOnEnvironment → exit(2). Bun.write(outPath, ...) is at :364, after the try/catch — no results file on the abort path. ${results.length} is accurate because results.push happens after the abort check. ✅

🟡 #2 — fails closed correctly; the compatibility concern is moot.
All three failure modes throw (non-zero exit :173, unparseable JSON :180, empty list :187). The call-site short-circuit is gone — :217 no longer has status.databasePath &&, so an empty path cannot skip the check and cannot even reach it. The "older CLI now hard-fails" risk does not exist: database list and model status both trace to 1a167e91 (ADR-053, 2026-07-09). No CLI has one without the other.

🟡 #3 — the fallback does not reintroduce the q4km/q8 confusion.
This is what I most expected to find broken. It holds. The fallback is only reachable when active_model_id is None and the daemon reports a resolved GGUF path — a state where the id is genuinely unavailable and no check can confirm the exact build. Degrading to "matched by filename, exact build unconfirmed, plus a warning saying so" is correct when the authoritative signal is absent. console.error is the right channel (scripts/ is exempt from the createLogger rule).

🟢 #9 — the guard-on-a-guard is not vacuous.
Selection keys on sites (literal sites seen), not prompts (successfully parsed). That distinction is what makes it safe:

1. Parser stops recognising prompt: → sites == 0 everywhere → zero files → !eval_scripts.is_empty() fires.
2. Parser sees sites but cannot parse literals → files still selected → per-file assert_eq!(prompts.len(), sites) (:376-383) fires, with a better message.
3. Non-fixture helper added → sites == 0 → skipped, not failed. The intended improvement, and it cannot mask a real fixture.

Failure modes are disjoint, each has a live assertion, and assert!(checked > 0) at :414 is a third backstop. The two-value (prompts, sites) return is precisely what makes content-based selection safe — nice design leverage.

Fixture claim independently verified. Whitespace-normalized comparison against de6413d0: routing.ts byte-identical modulo whitespace; the other two differ only in trailing commas and argument wrapping. No character inside any quoted string changed across all 12 matrix and 15 routing prompts. bunx prettier --check scripts/eval/ clean.

Other confirmations: EXIT_USAGE = 64 (EX_USAGE) at both sites and documented. timeoutMs/log doc comments accurately describe the cross-process contract — right call vs. deleting fields that document a real interface. CLAUDE.md note explains why. tracing::warn! includes error = %e and preserves degrade-don't-fail.

Unprompted catch, noted approvingly: abortOnEnvironment previously claimed "No scenarios were run" — false once a mid-run abort exists. Now reads "No results file was written." Tracing that the new abort path invalidated an existing message is easy to skip.

Tests re-run: cargo test -p nodespace-agent --lib guidance → 11/11 (incl. the anti-vacuity pin). bun run test:scripts → 21/21.

---

🟢 Nits

• 📁 packages/agent/src/agent_guidance.rs:346 — eval_prompts(&source).1.gt(&0) reads awkwardly; > 0 scans faster. Cosmetic.
• 📁 scripts/eval/preflight.ts:260-265 — the fallback-match warning is stderr-only. A modelMatchedByPath: true in provenance would let the caveat outlive the terminal session. Low value; provenance.model being a path is already a hint.

---

Recommendation

APPROVE

Both blocking items are addressed, and addressed well rather than minimally. 🔴 #1's fix distinguishes "never reached the model" from "called no tools" at the type level rather than string-matching the reply, preserves single-failure-as-real-failure, resets correctly, and routes through existing plumbing. 🟡 #2 fails closed on all three paths, and the compatibility concern turns out to be moot.

Both claims I probed hard hold: fixture changes are formatting-only (verified by normalized comparison, not by reading the diff), and the #9 content-based selection is not vacuous.

One residual 🟡 A: the sendFailed abort guards the scored turn but not priorTurns. Same class as the 🔴, smaller blast radius, one helper closure to fix. Worth taking in this PR if quick, but not holding the merge — the cited-number hazard that motivated the original REQUEST CHANGES is genuinely gone.

No project standard violated: no any, no lint suppression, no backward-compat scaffolding, no #[allow(dead_code)], no issue numbers in doc content or code comments, Bun-only, docs in nodespace-docs.

[malibio]

Re-review residual addressed — a61c0270

🟡 A — priorTurns not covered by the consecutive-failure guard. Fixed exactly as suggested: hoisted the counting into a turn() helper that both call sites go through, so every send counts against the budget rather than only the scored turn.

Verified live on the path the re-review identified. Ran the routing eval, waited until it reached a [context] turn, then killed the daemon:

=== EVAL EXIT: 2 ===
  7 scenario(s) had been scored; they are NOT written, ...
  No results file was written: a score from this environment would not mean anything.
--- results file? ---
ls: /tmp/ns-prior/out.json: No such file or directory

Daemon was isolated via NODESPACE_HOME (serving: /tmp/ns-prior/.nodespace/database/nodespace.db), so no live data was touched.

Both nits also taken:

• eval_prompts(&source).1.gt(&0) → > 0
• A fallback filename match now records modelMatchedByPath: true in the results file's provenance, so the "exact build unconfirmed" caveat outlives the terminal session that saw the stderr warning

Tests: test:all exit 0 · test:scripts 21/21 · nodespace-agent guidance suite 11/11 · pre-push gate (incl. e2e) passed.

All 13 original findings + the 1 re-review residual + 2 nits are now either addressed or deferred with a stated reason (#4 tracked as #1779, #7 blocked on #1764, #8 YAGNI, #10 tolerable by construction).

[malibio]

Clean end-to-end verification run ✅

Rebased onto 8c4c091d (clean — the one new commit touches only frontend files), test:all green on the new base, then ran the eval end to end against a freshly built daemon and a purged, isolated database.

Result: 9/12 — reproduces the honest baseline exactly

   Label:    final-verify
   Model:    gemma-4-e4b-q4km
   Context:  n_ctx 32768 · host RAM 17.2 GB
   Commit:   dbd54200
   Passed:   9/12

  ✓ 1  ✓ 2  ✓ 3  ✗ 4  ✓ 5  ✗ 6  ✓ 7  ✓ 8a  ✓ 8b  ✗ 8c  ✓ 8d  ✓ 8e

The three failures are scenarios 4, 6, and 8c — the same three as PR #1763's decontaminated run, and exactly the defects tracked in #1775 (4 and 8c) and #1637 (6). An independent reproduction of the known-honest baseline is good evidence the ported harness scores identically to the originals.

Provenance recorded (AC #5)

{
  "model": "gemma-4-e4b-q4km",
  "recordedAt": "2026-07-28T16:50:11.762Z",
  "hostMemoryGb": 17.2,
  "nCtx": 32768,
  "evalCommit": "dbd54200",
  "dirty": false
}

dirty: false — clean tree, so this number is fully attributable. Per-scenario failure strings preserved, which is what #1764 asks for.

Baseline comparison (AC #4)

All four transition states exercised against a synthetic prior run:

   REGRESSION  4
   REGRESSION  6
   FIXED       7
   NEW         8e → pass
   REMOVED     phantom
   regressions: 2

Environment discipline held

Daemon logged served_db_path=/tmp/ns-final/.nodespace/database/nodespace.db and seeded fresh (created_roots=27, not skipped) — so this measured current compiled guidance, not stale seeded prompts. Confirmed 0 nodes written to the real user database during the run.

Every acceptance criterion is now verified by execution rather than inspection.