[tfolkman]

Current review status

Ready for review at 3643a05, rebased onto upstream da20ac5. Fresh typecheck/build/npm test and whitespace checks pass: 1,676 server pass /16 skip; 971 frontend pass; 11 i18n checks pass. These are local checks, not a claim of hosted CI or a new production deployment. Merge #361 before #362; history includes the identical live commit. See the latest validation comment for boundaries and details.

---

Draft: read TensorFold's shared cache from the engine, and stop presenting its per-stream ratio as pool fill.

What it adds

Collector (server/collectors/LlmProbe.js) parses the optional live.cache group into a validated
observation with six states:

state	meaning
full	pool and an enabled prompt-state store, consistent
partial	one of them (a disabled store is partial, not absent)
unavailable	the engine publishes no group (older build) — not a measured zero
inconsistent	the engine itself says its accounting does not add up; no values
unsupported	a schema version this dashboard does not read; no values
invalid	the group was not an object

The four pool categories are accepted only when they sum to the reported capacity, and the store's
charged bytes only within its budget. A failed health read clears the previous observation, so a
stale pool is never presented as current. The same /metrics read also keeps the cumulative TTFT /
latency histogram buckets for the history sampler (a separate PR). Nothing here uses
kv_cache_usage_ratio.

Live panel (LlmCachePanel) is TensorFold-only: a partition bar (active / kept prefixes /
reclaimable growth / unclaimed) with the capacity, the per-request limit, the streams, lifetime
token reuse, and a detail block that separates the things that used to be one number: pool
invalidations vs placements refused, snapshot evictions/refusals, lookups, and failures by stage
(keep / restore / peer forget) beside the legacy total. An engine without the group gets one sentence,
not a row of zeros.

Overview chip shows % active · % kept from the same accounting, with the pool size, per-request
limit, streams and store budget in its tooltip.

VRAM breakdown: TensorFold's kv_cache_usage_ratio no longer colours the legend or raises
"Engine KV pool nearly full" — that ratio is one stream's share of its own window, not the shared
pool's occupancy. Other backends keep exactly their previous behaviour, and the legend says "not
reported by TensorFold" instead of guessing.

Verification

• npm run typecheck, npm run build, npm run test:frontend (817 tests), npm run test:server
  (1457 tests), npm run i18n:check, git diff --check — all pass.
• New tests: group validation (full / unavailable / inconsistent / unsupported / over-budget /
  disabled store / failed read clears), the panel's states, the overview chip, and the VRAM verdict
  for TensorFold vs an engine whose ratio is pool occupancy.

Compatibility: the endpoint snapshot only gains optional fields, and every field stays absent for
backends that do not report the group. The history PR builds on this one.

[MiaAI-Lab]

The panel reads the engine's own histograms (hist95 on the TTFT/latency series) and the token-weighted reuse ratio, which is the honest denominator — a mean of per-sample percentages would overcount light minutes. Two asks: state the restart behaviour (a prompt window spanning a restart must not claim a reuse share — #362's note suggests you handle it; a test would pin it), and rebase onto current main where the TFOT prefill split has since landed in LlmProbe. The UI addition itself is a product call, so leaving open for a maintainer look.

(Review notes prepared with Claude Code on behalf of the maintainer.)

[tfolkman]

Both points are in c31c804 (rebased on main at 7c2b963; the rebase picked up the TFOT prefill split without a semantic conflict).

Restart behaviour, pinned by a test. LlmProbe._applyTensorFoldCache now compares the group's engine_epoch with the last one it saw. On a change it calls _cacheRestarted(): the cached-prompt total and the prefill-split baselines are dropped, so the live tile reads — instead of dividing a cold process's prompt tokens by the previous process's cache. A transient failed /health is not a restart (the epoch is kept), and the same epoch on the next poll clears nothing.

server/collectors/__tests__/LlmProbe.tensorfold.test.js — "cache: a window spanning an engine restart claims no reuse share": 900k/1M before the restart reads as 90%; the same epoch again keeps it; a new epoch drops the cached total and the hit rate; a null read in between changes neither.
src/components/SparkPage/LlmCachePanel.test.tsx — "claims no reuse share for a window that spans an engine restart": the tile shows — and no percentage.

Totals from the new process still replace the old ones when it reports them, so the panel recovers on the next poll. History is unaffected here — it already rebaselines on the epoch (in #362).

Checks after the rebase: npm run typecheck clean, npm test 1470 server tests (1454 pass, 16 skip, 0 fail), vitest run 831 pass, npm run build and i18n:check clean, git diff --check clean.

[tfolkman]

Device-validation update; no engine-pin or inference-policy change:

• The pinned telemetry integration has completed a normal four-rank CUDA rollout and over 28 hours of ordinary-workload observation, without synthetic warming or forced eviction. The recorded capture contains 981 completions, 76,669,208 prompt tokens and 71,909,376 cached tokens: 93.79% token-weighted reuse. All 13 history counter deltas reconcile exactly to the stored native-counter baselines; no recorded cache failures or placement blocks. This is not a hardware-soak qualification.
• Bounded deterministic cold/replay/append SSE, three tool-choice flows, and client-close/reservation-drain checks pass. A 24-request polling on/off/off/on comparison measured median TTFT +1.46% and stream-decode −0.63%, below the predeclared 5% investigation threshold. This small comparison is not a general performance guarantee; the three-request old/new-engine comparison remains inconclusive.
• Real history survives a dashboard-only restart. All five ranges and desktop/390px rendering pass; missing/partial/reset cases use explicitly labelled replay fixtures. The sidecar is about 2.44 MB at this observation length, versus the previously measured 7,866,328-byte saturated one-port fixture. Native interval conservation and stored aggregate sums remain exact; independently rounded query means can differ by one position in their displayed category sum (four rounding errors are bounded by two positions).
• Fresh deployed-integration checks: recipe 182 pass; dashboard typecheck/build and full npm test pass (1,356 server pass / 16 skip, 753 frontend pass, plus 11 collector checks). The approved image build previously ran 41 CPU tests, including the interval test. Known native full-suite baseline failures remain failures, not relabelled green. Exact allocation-event profiling and actual candidate rollback were not performed.

No production lifecycle controls, automatic tuning, new model requests, or private deployment material are added by these PRs. Draft/merge decisions remain with the maintainers.

[tfolkman]

Restart semantics are pinned in the dependent history stack (#362): a new engine epoch seeds a fresh baseline and does not claim reuse for a prompt window spanning the restart; the new observation-identity test also covers a repeated timestamp with a changed epoch. The live collector clears its reuse window on epoch change. Follow-up 3738afd adds visible restart/reset/gap markers and pressure coverage, with failing-before/passing-after backend/UI regressions. Current-main/TFOT rebase remains separate upstream integration work; the matched deployment checks are not evidence that a newer main has been device-qualified.

[tfolkman]

Rebased onto current upstream main (da20ac5), retaining its translated parallel-request tooltip. Review head: 3643a05.

Fresh checks on this exact head:

• git diff --check, npm run typecheck, npm run build: pass.
• npm test: 1,676 server tests pass, 16 skip; 971 frontend tests pass; 11 i18n checks pass. No failing tests.
• Added-diff scan found no private host paths, network identifiers, deployment epoch or private-key markers. This scan is a specific check, not a general security certification.

This is a source-only review refresh. The earlier bounded live-device validation applies to the deployed pinned integration, not a claim that current upstream main was deployed. No production lifecycle action or new model workload was performed for this refresh.

The history PR #362 includes this live commit; merge #361 first, then review/merge the remaining history diff. Both branches now preserve current upstream localization and have fresh test results.