Skip to content

perf(seo): serve the trending boards from Workers Cache - #329

Merged
harlan-zw merged 9 commits into
perf/cacheable-htmlfrom
perf/edge-cache-trending
Sep 30, 2026
Merged

harlan-zw merged 9 commits into
perf/cacheable-htmlfrom
perf/edge-cache-trending

Conversation

@harlan-zw

@harlan-zw harlan-zw commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on #328.

❓ Type of change

  • 👌 Enhancement

📚 Description

#328 made public HTML the same for every visitor. This turns Workers Cache on for one page family, /skills/trending and its ?range= boards, so we can watch it in production before skill, repo and category pages follow. Every other page keeps the private, no-store default.

The rule is edgeCache({ maxAge: 60, staleWhileRevalidate: 3600 }):

  • 60s fresh. The board's feed (/api/feed/trending) already caches for 300s, so the edge adds at most a minute on a busy colo.
  • 3600s stale. The inputs move every 15 minutes at most (sync-x-mentions) and hourly (social mentions, X engagement). A quiet colo serves one copy up to an hour old and refreshes it in the background, instead of rendering on every visit.
  • 3660s total, well inside the 36000s nuxt-skew-protection keeps old chunks for, so nuxt-cloudflare keeps the value unclamped. Browsers still get no-store and ask the edge each time.

The Workers Cache key is the path plus the query string (and the Worker version), so week, month and all never share an entry. Parameter order matters, so a reordered or padded query is its own entry and its own render.

One ordering bug showed up on a local build. nuxt-cloudflare and nuxt-skew-protection both judge each HTML response in beforeResponse, in module order. nuxt-cloudflare ran first, saw the __nkpv version cookie that skew protection was about to drop, and marked every browser navigation no-store. Only requests without Sec-Fetch-Dest: document (crawlers) would have been stored, and Vary: Sec-Fetch-Dest keeps browsers on a separate variant, so browsers would never have hit. nuxt-skew-protection now loads before @harlan-zw/nuxt-cloudflare. That ordering is implicit; it probably wants fixing in nuxt-cloudflare so the cookie check runs after other modules' hooks.

The key has no host, and www.skilld.dev routes to the same Worker since #316. A route rule's cache headers are on the event before the canonical host middleware sends its 301, so a stored www redirect could have answered the apex URL and looped. That redirect now always says no-store. I checked production first: the www 301 on /api/clusters already answers cf-cache-status: BYPASS, but I could not tell why, so I'd rather not rely on it.

pnpm production:edge-cache (scripts/check-edge-cache.ts) proves it against the deployed site, per board:

  • a repeat anonymous browser request comes back cf-cache-status: HIT, STALE or UPDATING (up to 12 tries, since the key includes the new Worker version)
  • no anonymous response sets any cookie, and Vary is exactly Accept, Sec-Fetch-Dest
  • Accept: text/markdown still gets the 307 to /skills/trending.md
  • a request carrying a nuxt-session cookie gets the anonymous payload and loadStrategy: client-only, and no new session cookie

The deploy workflow runs it after production:deploy. It fails the job but does not roll back. The rollback smoke re-runs against the previous version, and a version from before this rule could never pass it, so it can't live there.

Rollout: watch the deploy's edge cache step and cf-cache-status on trending for a week. If it holds, a follow-up adds the same rule to skill, repo and category pages and extends EDGE_CACHED_PATHS. Those pages render more per-skill state, so I'd check each one's payload for anything request dependent first.

nuxt.config.ts overlaps #320, #322, #323, #325 and #326. Only #320 conflicts, on the same nitro.handlers line as #328. scripts/lib/production-smoke.ts is untouched.

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

Workers Cache keys on the path and query, never the host, and www routes
to the same Worker. By the time the canonical host middleware runs, a
route rule's edge cache headers are already on the response, so a cached
page's www 301 could be stored and served to the apex URL, which would
redirect to itself. The redirect now always says no-store.
wrangler dev has no Workers Cache, so nothing local can show that a page
is stored or that storing it is safe. pnpm production:edge-cache checks
the deployed trending boards: a repeat anonymous visit is a cache hit, no
anonymous response sets a cookie, Accept: text/markdown still gets the
307, and a request with a session cookie gets the anonymous render.

The deploy workflow runs it after the rollback-gated smoke. It fails the
job without rolling back, because the rollback smoke re-checks the
previous version, which has no cache rule to pass it with.
The first HTML in the edge cache, to prove the setup on one page family
before skill, repo and category pages follow. 60 seconds fresh adds at
most a minute to a feed that already caches for five. An hour of stale
while revalidate lets a quiet colo serve one old copy while it refreshes,
since the board's inputs move every 15 minutes at most.

nuxt-skew-protection now loads before nuxt-cloudflare. Both judge each
HTML response in beforeResponse, in module order. With nuxt-cloudflare
first it saw the __nkpv version cookie that skew protection was about to
drop and marked every browser navigation no-store, so only crawlers
could have filled the cache.
@harlan-zw
harlan-zw added this pull request to stack #330 September 30, 2026 06:37
@harlan-zw

Copy link
Copy Markdown
Collaborator Author

🤖 Agent verification on a local build:test preview (wrangler dev, which has no Workers Cache).

  • Before the module order change, a browser navigation to /skills/trending?range=month answered Set-Cookie: __nkpv=test and cloudflare-cdn-cache-control: no-store. After it: no Set-Cookie, cloudflare-cdn-cache-control: public, max-age=60, stale-while-revalidate=3600, Vary: Accept, Sec-Fetch-Dest.
  • The same request with a nuxt-session cookie answers cloudflare-cdn-cache-control: no-store and the payload with loadStrategy: client-only.
  • EDGE_CACHE_BASE_URL=http://localhost:8787 pnpm production:edge-cache passed every check except the hit: all 13 tries per board reported no cf-cache-status, as expected without Workers Cache. The hit is only provable after deploy.
  • The www redirect guard is covered by a unit test only; wrangler dev ignores an overridden Host.

A failing check skipped the later DX report steps, which run on success by default. Running it last keeps them.
A hung edge blocked the deploy job until the 20 minute limit.
A failed fetch still rendered HTTP 200, so the edge would store the error state and its noindex meta.
@harlan-zw

harlan-zw commented Sep 30, 2026 •

Copy link
Copy Markdown
Collaborator Author

🤖 REVIEWING · 35% · Git worktree ready

Harlan Agent Kit posted this automated review. AI open source policy. Last updated: 2026-09-30 08:59 UTC.

Next: Review the diff.

  • Stop Review and any follow-up repair

Leave unchecked to finish Review after merge.

@harlan-zw harlan-zw added the harlan-agent-pending The automated Review is waiting on a gate for this head commit. label Sep 30, 2026
@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. and removed harlan-agent-pending The automated Review is waiting on a gate for this head commit. harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Sep 30, 2026
@harlan-github-agent harlan-github-agent Bot added harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. harlan-agent-running An Agent holds a Task on this issue or pull request right now. and removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. labels Sep 30, 2026
The check now fails when two boards render the same title and canonical URL, and retries a transient non-200 once.
@harlan-github-agent harlan-github-agent Bot removed the harlan-agent-running An Agent holds a Task on this issue or pull request right now. label Sep 30, 2026
@harlan-github-agent

harlan-github-agent Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

🤖 MERGED

Harlan Agent Kit posted this automated review. It is not Harlan's personal review or approval. AI open source policy. Last updated: 2026-09-30 10:12 UTC.

GitHub merged this pull request.

  • Logged (78/100): The post-deploy check lists '/skills/trending?range=month' and '/skills/trending' as separate cached paths, but month is the default range so both URLs render an identical title and canonical, collide in pageIdentity, and the check reports View code
    • Ask an agent to verify and fix this finding
  • Logged (18/100): requestAnonymous retries a non-200 response once but propagates a thrown network error immediately, so one transient fetch failure fails the board and reddens the deploy job for idempotent GET traffic. View code
    • Ask an agent to verify and fix this finding

Review completed. No finding above 80/100.

Selected findings run after merge and open separate pull requests.

@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. and removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Sep 30, 2026
@harlan-zw
harlan-zw merged commit a48076d into main Sep 30, 2026
6 checks passed
@harlan-zw
harlan-zw deleted the perf/edge-cache-trending branch September 30, 2026 09:32
@harlan-github-agent harlan-github-agent Bot removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant