Repository navigation
Conversation
TensorFold's /metrics publishes tensorfold:prompt_tokens_cached_total on builds that keep prompt states (the four-node DeepSeek build, and the two-node one after its metrics port), but _applyTensorFoldMetrics only used prefix_cache_queries_total and prefix_cache_hits_total for the hit rate. The token ledger therefore had no cached counter and the token pages showed the cache as unknown. Read prompt_tokens_cached_total in the same scrape as prompt_tokens_total and set totalCachedTokens from it, falling back to prefix_cache_hits_total when the dedicated counter is absent. Clamp the cached count to that scrape's prompt total so it can never exceed the prompt it belongs to. Builds that publish neither counter keep their previous value, and the queue-only poll and the /health cached_tokens_total path are unchanged. Add a regression test covering the direct counter, the fallback, the clamp, absent counters, and the queue-only poll.
Owner
|
The cache-counter change is now on main as fa5c0e4, preserving the original author and cherry-pick attribution. The collector reads the dedicated counter, uses the prefix-cache fallback, and clamps against the same scrape's prompt total ( |
pavelkubicek
added a commit
to pavelkubicek/sparkDash
that referenced
this pull request
Oct 27, 2026
78 upstream commits: MQTT publishing (MiaAI-Lab#248), harness occupancy helpers (MiaAI-Lab#208), GPU clock cap from Settings (MiaAI-Lab#218), settings i18n foundation (MiaAI-Lab#226), configurable GPU temperature limits (MiaAI-Lab#273), Locals/Splash engines, TensorFold cache-counter fixes (MiaAI-Lab#252/MiaAI-Lab#260/MiaAI-Lab#272) and the zsh-safe remote probes (MiaAI-Lab#255/MiaAI-Lab#257). Conflict resolutions: - server/collectors/LlmProbe.js + its TensorFold test: took main's implementation, which supersedes this fork's /health `live` + /metrics work with the cache-counter precedence, running/waiting aliases, histogram validation and prefill holds. This drops the fork's parallel /metrics fetch (micro-optimisation only). - SystemCollector: main's zsh-safe find probes merged into the fork's build/parse refactor, keeping the fork-only RAPL package-power reads; _getRemoteNetwork now uses the injected executor (the merge had left the fork body calling sshExec, which made main's zsh-glob corpus test run real SSH). - OverviewPage: FleetKpis keeps the fork's placement but takes all sparks, so hidden cluster workers count in the KPI tiles (main's fix). - SparkPage: main's column layout (History in the hardware column, storage/network in the right column) over the fork's full-width History panel. - SettingsDialog: fork VulnHunter/REA opt-in toggles kept, metrics-history toggle switched to main's i18n keys. - index.js: fork vulnHunt/rea spark cleanup kept alongside MQTT removeUnit/applySettings. - CHANGELOG/README/package.json: both sides' entries kept; test globs unioned so the fork-only models/autopower/vulnhunt suites still run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
_applyTensorFoldMetricsnow readstensorfold:prompt_tokens_cached_totalandsets
this.totalCachedTokensfrom it. When that counter is absent, it fallsback to
tensorfold:prefix_cache_hits_total, which carries the same number. Thecached counter and
prompt_tokens_totalcome from the same scrape, and thecached value is clamped to that scrape's prompt total so it can never exceed the
prompt it belongs to.
Why
TensorFold builds that keep prompt states publish the cached prompt split in
/metrics, but the collector only used the prefix-cache counters for the hitrate. The token ledger therefore had no cached counter, and the token pages
showed the cache as unknown.
How
The new read sits in the existing non-queue branch of
_applyTensorFoldMetrics, beside the prompt and generation totals. Builds thatpublish neither counter keep their previous value. The
/healthpath(
cached_tokens_total) and the queue-only poll are unchanged.Testing
npm ci,npm run typecheckandnpm run build: passed.node --test server/collectors/__tests__/LlmProbe.tensorfold.test.js: 45/45 passed.node --test server/llmtokens/__tests__/*.test.js: 26/26 passed.test:serverfile list, Node 26 on macOS):maingives 1128 tests with 20 failures; this branch gives 1129 tests with the same 20 failures, all inserver/llmlaunch/__tests__(launcher, api) andserver/tooleval/__tests__(tooleval, api), which this change does not touch.The regression test covers the direct counter, the prefix-cache fallback, the clamp, absent counters, and the queue-only poll.