Skip to content

fix(citable): stop poisoning aggregator cache with all-draft set - #607

Merged
Flotapponnier merged 1 commit into
devfrom
fix/citable-aggregator-poisoning
Jun 21, 2026
Merged

fix(citable): stop poisoning aggregator cache with all-draft set#607
Flotapponnier merged 1 commit into
devfrom
fix/citable-aggregator-poisoning

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

Fixes /api/citable returning all benchmarks as draft when /api/stat/
for the same bench returns live data. Root cause: loadAllBenchmarksCached
was returning an all-draft set whenever per-bench fetches threw at cold
start, and the cache then served that poisoned set for the rest of the
revalidate window.

This change restores the throw-on-all-draft behaviour at the cache
boundary so unstable_cache keeps the previous good value during a Prom
blackout. The known call sites are wrapped explicitly: API endpoints and
feeds (citable, llm-context, llms.txt, rss.xml) return 503 with a
retry-after hint, pages and shared loaders (home, benchmarks index,
bench slug, category, mcp page, chains, providers) use a new
getBenchmarksSafe wrapper that renders draft placeholders, and the
sitemap already uses safeLoad which catches and falls back.

Cache key bumped from v12 to v13 to flush any poisoned snapshot already
stored in Upstash KV.

Adds a regression test that asserts the aggregator never returns a
stable all-draft list. The aggregation logic is extracted into a pure
aggregateBenchmarks function so the test does not need Prometheus or
unstable_cache.

Verification

Before:

  • /api/citable: every bench reports status draft, value null, leader null
  • /api/stat/bridge-quote-latency: status live, value 298.7, leader Mobula

After (locally, via the regression test): aggregator throws
AllBenchmarksDraftError when every loader fails or returns undefined,
returns the partial list when at least one bench is live, never returns
a stable all-draft set.

Test plan

  • pnpm typecheck
  • pnpm lint (no new warnings)
  • bun test src/lib/ (83 pass)
  • pnpm build (no crash on the all-draft path either)
  • After merge to dev: curl preview /api/citable, confirm live values
  • Cherry-pick to main once preview looks healthy

Fixes /api/citable returning all benchmarks as draft when
/api/stat/<slug> for the same bench returns live data. Root cause:
loadAllBenchmarksCached returned an all-draft set whenever per-bench
fetches threw at cold start, and the cache then served that poisoned
set for the rest of the 60s revalidate window.

Restores the throw-on-all-draft behaviour at the cache boundary so
unstable_cache keeps the previous good value during a Prom blackout.
Adds an AllBenchmarksDraftError sentinel and a getBenchmarksSafe
wrapper. Call sites split:

- API endpoints and feeds (citable, llm-context, llms.txt, rss.xml)
  catch the sentinel and return 503 with retry-after, so downstream
  consumers do not treat the placeholder set as ground truth.
- Pages (home, benchmarks index, bench slug, category, mcp page) and
  shared loaders (chains, providers) use getBenchmarksSafe which
  catches the sentinel and renders draft placeholders so the build
  and the page render still succeed.
- Sitemap already uses safeLoad which catches and falls back.

Cache key bumped from v12 to v13 to flush any poisoned snapshot
already stored in Upstash KV.

Adds a regression test that asserts the aggregator never returns a
stable all-draft list. Extracted aggregateBenchmarks as a pure
function so the test does not need Prometheus or unstable_cache.
@Flotapponnier
Flotapponnier merged commit 69d14b2 into dev Jun 21, 2026
1 check passed
@Flotapponnier
Flotapponnier deleted the fix/citable-aggregator-poisoning branch June 21, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant