Repository navigation
fix(registry): serve tag profiles stale-while-revalidate - #190
Merged
Merged
Conversation
harlan-github-agent
Bot
added
harlan-agent-running
Contributor
Author
🤖 MERGED
GitHub merged this pull request. No material findings were recorded. 17de384c-fcc2-4cad-9221-b58b6eb3a982 |
harlan-github-agent
Bot
added
harlan-agent-blocked
harlan-github-agent
Bot
added
harlan-agent-running
1 task done
harlan-github-agent
Bot
removed
the
harlan-agent-ready
harlan-zw
added a commit
that referenced
this pull request
Sep 14, 2026
* perf(registry): cut skilld-db rows read on repo skill counts skilld-db reads grew from 104M to about 1B rows a day, pushing the account past the 25B/month D1 free tier. The hot statements computed a correlated repo Skill count per row before any LIMIT, reading sum(c^2) rows, and re-ran on one to five minute caches. repos.repo_skill_count is not used: sync stores the SKILL.md file count and never updates it when source_resolved clears, and it disagrees with the live count for 7,159 of 8,618 repos. Claude-Session: https://claude.ai/code/session_014N85FDJLggfBs3BHJXT7GZ * refactor(registry): leave handler cache lengths to open swr PRs PRs #189 and #190 already change the tag and cluster handler cache options, so this branch keeps only the query changes there. Claude-Session: https://claude.ai/code/session_014N85FDJLggfBs3BHJXT7GZ --------- Co-authored-by: Harlan GitHub Agent <harlan-github-agent[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Linked issue
Closes #185
❓ Type of change
📚 Description
The sentry-checkin routine flagged that
/api/tags/[slug]recomputes on every crawler sweep. The handler cached for only 60 seconds with stale-while-revalidate disabled, and the backing query is a heavy FTS scan plus per-row subqueries, so a sweep across many tags re-ran it for every page every minute. That matches the pattern behind SKILLD-19, SKILLD-1C, and SKILLD-1H.The profile now stays fresh for an hour and keeps serving stale for a day while a background refresh runs, the same shape the skill-live endpoint already uses. A stale entry also survives a failed refresh, so a tag whose data briefly empties out keeps rendering its last good profile instead of erroring mid-sweep.
One thing to check:
fetchedAtin the payload now shows when the cached profile was computed and can lag up to a day. Nothing in the UI or sitemap seems to read it, but worth a look.