Skip to content

hotfix(citable): stop poisoning aggregator cache with all-draft set - #610

Merged
Flotapponnier merged 1 commit into
mainfrom
hotfix/citable-aggregator-poisoning
Jun 21, 2026
Merged

hotfix(citable): stop poisoning aggregator cache with all-draft set#610
Flotapponnier merged 1 commit into
mainfrom
hotfix/citable-aggregator-poisoning

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

Cherry-pick of #607 to main as a hotfix.

Fixes /api/citable returning all benchmarks as draft when /api/stat/
for the same bench returns live data. Root cause: loadAllBenchmarksCached
was returning an all-draft set whenever per-bench fetches threw at cold
start, and the cache then served that poisoned set for the rest of the
revalidate window.

This change restores the throw-on-all-draft behaviour at the cache
boundary so unstable_cache keeps the previous good value during a Prom
blackout. The known call sites are wrapped explicitly: API endpoints and
feeds (citable, llm-context, llms.txt, rss.xml) return 503 with a
retry-after hint, pages and shared loaders (home, benchmarks index,
bench slug, mcp page, chains, providers) use a new getBenchmarksSafe
wrapper that renders draft placeholders, and the sitemap already uses
safeLoad which catches and falls back.

Cache key bumped from v12 to v13 to flush any poisoned snapshot already
stored in Upstash KV.

Adds a regression test that asserts the aggregator never returns a
stable all-draft list.

Cherry-pick notes

Some conflicts resolved against main's slightly older surface:

  • src/app/benchmarks/category/[cat]/page.tsx is deleted on main, dropped
    from the cherry-pick.
  • src/app/api/citable/route.ts does not have citeBundle on main, kept
    the existing field set, only added the AllBenchmarksDraftError import
    and the try/catch + 503.
  • src/lib/providers.ts isAll import not needed on main, dropped.
  • src/app/benchmarks/[slug]/page.tsx kept main's lucide-react + import
    surface, only switched the loader to getBenchmarksSafe.

Test plan

  • pnpm typecheck
  • pnpm lint (no new warnings vs main baseline)
  • bun test src/lib/ (34 pass on main worktree, including the new regression tests)
  • pnpm build
  • After merge: vercel --prod from main checkout, then curl prod /api/citable

Fixes /api/citable returning all benchmarks as draft when
/api/stat/<slug> for the same bench returns live data. Root cause:
loadAllBenchmarksCached returned an all-draft set whenever per-bench
fetches threw at cold start, and the cache then served that poisoned
set for the rest of the 60s revalidate window.

Restores the throw-on-all-draft behaviour at the cache boundary so
unstable_cache keeps the previous good value during a Prom blackout.
Adds an AllBenchmarksDraftError sentinel and a getBenchmarksSafe
wrapper. Call sites split:

- API endpoints and feeds (citable, llm-context, llms.txt, rss.xml)
  catch the sentinel and return 503 with retry-after, so downstream
  consumers do not treat the placeholder set as ground truth.
- Pages (home, benchmarks index, bench slug, category, mcp page) and
  shared loaders (chains, providers) use getBenchmarksSafe which
  catches the sentinel and renders draft placeholders so the build
  and the page render still succeed.
- Sitemap already uses safeLoad which catches and falls back.

Cache key bumped from v12 to v13 to flush any poisoned snapshot
already stored in Upstash KV.

Adds a regression test that asserts the aggregator never returns a
stable all-draft list. Extracted aggregateBenchmarks as a pure
function so the test does not need Prometheus or unstable_cache.
@Flotapponnier Flotapponnier added the hotfix Production hotfix label Jun 21, 2026
@Flotapponnier
Flotapponnier merged commit 6f11ea3 into main Jun 21, 2026
1 check passed
@Flotapponnier
Flotapponnier deleted the hotfix/citable-aggregator-poisoning branch June 21, 2026 12:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hotfix Production hotfix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant