[ACTION NEEDED] Daily check-in: skilld-dev/skilld.dev

Harlan Agent Kit maintains this report automatically. Each run remains in the private archive.

[ACTION NEEDED] Daily check-in: 2026-10-11

AMBER skilld.dev

Live health is clear, but two D1 Sentry incidents recurred.

Check	Result	Detail
baseline	Pass	Fresh, 14-hour gap
git	Pass	`5dfb8de1` on expected ref
deploy	Pass	Worker matches `5dfb8de1`
CI	Pass	Test, Deploy, Weekly social passed
home / skills	Pass	Both HTTP 200
protected health	Pass	Fresh, no warnings
workers	Pass	No non-OK outcomes
analytics	Pass	1 run copy, 0 install copies
database	Warn	26 terminal submission jobs
Sentry	Warn	SKILLD-1T and SKILLD-3V recurred
signing key	Pass	38 days remain

Needs you

Review the D1 escalation draft. SKILLD-1T and SKILLD-3V recurred. Escalate recurring D1 internal errors to Cloudflare #393
Close Unpinned skills answer a permanent 503 retry loop on skills-raw #577. Its trigger is fixed by fix(registry): complete large repository source refresh #575 and stayed quiet.
Run the Week 1 SEO read from a controller with nuxtseo 0.5.8+. This worker lacks the CLI.
Rotate the signing key before 2026-11-20.

Changed since yesterday

Recovered: prior stale-reservation and memory-kill burst aged out.
New: D1 incidents recurred under existing Escalate recurring D1 internal errors to Cloudflare #393.

Unchanged work (3)

X spend (X discovery projects past the accepted $40 monthly spend line #179), $17.40/mo Post-only estimate, below target
Purpose-held review backlog, 322 rows, parked by fix(discovery): park purpose-held submissions in the ledger #605
Terminal submission outcomes, no re-enqueue evidence

Evidence

Exit 1 | coverage complete | deployment 5dfb8de1 | baseline fresh, 14h
Archive: /home/agent/.local/state/daily-checkin/skilld-dev/skilld.dev/2026-10-12T01-59-49-592Z-d6b86568-f44e-4076-af8d-866682fb291f.json

2026-09-02T21:40:00.000Z — Skipped. This run was due more than 6 hours ago, so it was skipped.

Daily check-in 2026-09-03

RED. The CI gate is broken on main: two Test runs and one Deploy run died at pnpm build with exit code 129 on the self-hosted runner, so the deploy gate froze and production has sat three commits behind since 2026-09-02 01:31Z.

Pulse

Metric	Value	Prior
Users	24	24
Skills	11,739 (+197 new)	11,542
Repos changed in window	62	48
Installs	0	6
Digests sent / failed	0 / 0	2 / 0
Known AI cost	$0.00 recorded	$0.00
X spend	$13.80/mo projected (25/2000 reads today)	$12.30/mo

Embeddings completed 61, none failed or stuck. One skill sync failure, two user-impacting repos newly broken (see Broken). send-digests is monthly (0 9 1 * *), so 0/0 is on cadence, not silence.

X reads today are 25 against a 2,000 daily target, 1% used, x_over_budget false. Projection is $13.80/month, under the $20 target, at 5 reads per verified skill. 192 posts kept against 25 billed reads, so the query is not paying for posts it discards. Steady state, no fault.

Overnight

Production is version 653b02f4 at 1f5dcb9 (docs-only), live since 2026-09-02T01:31:45Z. Four commits have merged since; none is deployed (see Broken).
Test state is failure on HEAD 4fcf0d9 (run 33715449270), and failed on 8fdb2b2c the night before (run 33587338442). Branch runs of the same workflow passed, so the gate break is environment-shaped, not code.
Deploy to Cloudflare state is failure, consecutive failures 1: the real deploy run for 827faafb failed (run 33583522347). The skipped runs after it are the guard reading through, as designed.

Broken

`pnpm build` killed with exit 129 on the self-hosted runner (new)

Three occurrences, all the same shape: the build job's pnpm build step ends with [ELIFECYCLE] Command failed with exit code 129 (SIGHUP).

Deploy run 33583522347 on 827faafb, 2026-09-02T02:36:54Z.
Test run 33587338442 on 8fdb2b2c, 2026-09-02T05:16:30Z.
Test run 33715449270 on 4fcf0d9, 2026-09-03T04:36:38Z.

The same build job passed on branch SHAs 3cc3ed8a (2026-09-02T02:56Z) and adf14055 (2026-09-03T04:11Z) in the same window, so this is not a code fault. Exit 129 is a hangup, not a compiler error. Hypothesis: memory pressure on the Hogwild runner kills the build process, and main merges have been landing while other work holds the machine.

Consequence: the deploy freeze is the user-facing harm. #119 (ungh tree retry), #112 (run-first copy) and #121 (digest summary pause) are all written but not serving. SKILLD-1E grew 3 events to 2026-09-02T14:54:02Z on GET /api/skill-files/tldraw/tldraw/review-docs precisely because production predates #119. No new defect there; the fix exists and is parked.

D1 overload crawler sweep, recurrence (watch row 2026-09-01)

Thirteen new Sentry ids, every event inside one five-second window, 2026-09-02T09:17:07Z to 09:17:12Z, userCount 1 on each, which is the crawler-traversal signature: SKILLD-1F 195 events on GET /api/skill-related/github/awesome-copilot/aws-resource-health-diagnose, SKILLD-1H 12 on GET /api/tags/test-automation?view=data, SKILLD-1J 7, SKILLD-1T/M/G 3 each, SKILLD-1R/Q/P/N 2 each, SKILLD-1S/K 1 each, all D1_ERROR: D1 DB is overloaded. The sweep's write side fired once: SKILLD-1V, KV PUT failed: 429, 1 event at 10:04:19Z on GET /api/skills/vercel-labs/agent-skills/react-best-practices, the same known family as SKILLD-C/SKILLD-17.

Verdict: this is the 2026-09-01 watch row recurring with fresh ids, not a second mechanism. Its exit condition, a crawler sweep that produces no burst, is unmet. The family stays on watch; no separate code change this pass. Workers nonOk is empty and the front door stayed 200 throughout.

Two user-impacting repositories newly broken (0 → 2)

newly_broken_repos_total is 8 and newly_broken_repos_impacted is 2. Named read-only: levi-qiao/skillmill, broken since 2026-09-02T15:01:08Z, holds skills; its single repo fetch 404 failed job at 15:01:09Z is the designed terminal shape the resolved ledger row wants, and is the health email's "1 skill recorded a new sync failure". aamarva/aamarva-adk, broken since 2026-09-02T20:02:13Z, also holds skills. Both are real activation-surface impact but small; the nightly email already carries them. Cumulative broken_repos at 2,090 is known inventory.

Not broken, stated explicitly

https://skilld.dev/ and /skills both 200.
workers.nonOk is empty: 80,649 successes, 39 client disconnects, no loadShed.
Migration heads match at 0117_email_cadence_split.sql, local and production. No drift, no uncommitted migrations.
missingExpectedTables is empty, no stale reserved jobs, no stale scheduled tasks.
17 of 18 scheduled tasks ok; only send-digests is partial, the known 09-01 AI-Gateway-credits window whose pause flag (chore(digest): pause the AI summary behind an explicit flag #121) is written but undeployed.
7 failed jobs: 6 no_supported_skill_paths (terminal submission decisions, not faults) and 1 repo fetch 404 (designed terminal shape). All ten recent job batches finished failed_jobs: 0.
Sentry probe complete, not truncated.

Drift and observability

Deploy SHA: production 1f5dcb9 vs HEAD 4fcf0d9, three behind, frozen by the gate break above.
The first probe run today degraded four probes (deploy, ci, d1, workers) on auth errors: the GITHUB_TOKEN exported from .env is expired, and CLOUDFLARE_API_TOKEN is only present commented out in .env. Worse, the expired token silently broke git fetch, so origin/main was stale and the first archive recorded false migration drift (local 0116 vs prod 0117). Rerun on a fresh fetch reads clean. The probe itself reads origin/main without fetching, so nothing caught the staleness.
Archives today: 2026-09-03.json (degraded probes), 2026-09-03.rerun-0559.json (stale-ref drift), 2026-09-03.rerun-0601.json (authoritative, the source of every number above).

Proposed actions

Make the CI build survive runner kills. Add a bounded retry to the pnpm build step in both Test and Deploy to Cloudflare, and keep the two workflows from building simultaneously on the Hogwild runner (concurrency group). This unfreezes fix(registry): retry a transient ungh tree read before answering 503 #119/fix(copy): align site copy with v3 run-first positioning #112/chore(digest): pause the AI summary behind an explicit flag #121 and closes the RED. Verify: merge, then Test on the merge SHA goes green with the build job passing, and Deploy to Cloudflare lands the SHA.
Rotate the expired GITHUB_TOKEN in .env. It broke four check-in probes and, via a failed git fetch, produced a false migration-drift archive. Local credentials only; no deploy needed. Verify: node scripts/tools/daily-checkin-data.mjs completes with zero probe auth errors.
Teach readMigrationState to fetch and to fail loudly. Fetch origin/main before git ls-tree and record a fetch failure as an explicit probe error, so a stale ref can never read as migration drift again. Verify: GITHUB_TOKEN=invalid node scripts/tools/daily-checkin-data.mjs reports a fetch error instead of drift.
Hold the D1-overload watch row open. No code change this pass. If the next crawler sweep repeats at this size on already-cached endpoints, revisit the sweep's burst profile.

The CI build job's pnpm build step dies with exit 129 (SIGHUP) on the self-hosted runner, which broke the Test and Deploy gates and froze production three commits behind main.

Target: .github/workflows/test.yml

Verify with: After the merge, env -u GITHUB_TOKEN gh run watch $(env -u GITHUB_TOKEN gh run list --branch main --workflow Test --limit 1 --json databaseId --jq '.[0].databaseId') --exit-status passes with the build job green, and Deploy to Cloudflare lands that SHA.

Estimated to change 2 files.

readMigrationState reads the origin/main ref without fetching it, so a silently failed git fetch produced a false migration-drift archive on 2026-09-03.

Target: scripts/tools/daily-checkin-observability.mjs readMigrationState

Verify with: ``GITHUB_TOKEN=invalid node scripts/tools/daily-checkin-data.mjs reports an explicit fetch-failure probe error instead of a migration drift verdict.

Estimated to change 1 file.

The GITHUB_TOKEN exported from .env is expired, which degraded four check-in probes and broke the git fetch behind a false migration-drift reading.

Target: .env

Verify with: ``node scripts/tools/daily-checkin-data.mjs --save completes with zero probe auth errors in deploy, ci, d1, and workers.

Estimated to change 1 file.

Daily check-in 2026-09-03

RED. The site serves and Workers are clean, but production is frozen 4 commits behind main for 44 hours with two fixes waiting, and the CI gate is unreadable because the check-in's GitHub credential answers 401.

Pulse

Metric	Value	Prior
Users	24	24
Skills	11,844 (+302 new)	11,542
Repos changed in window	115	48
Installs	2	6
Digests sent / failed	0 / 0 (monthly cron, next fire 10-01)	2 / 0
Known AI cost	$0.00 recorded	$0.00
X spend	$24.00/mo projected (99/2000 reads today)	$12.30/mo (6/2000)

Embeddings completed 134, none failed or stuck. Three skills recorded sync failures, all source-gone verdicts (see Broken). ai_submitted is 0 with the generation lanes quiet; the digest summary pause (#121) is merged but undeployed.

X projection nearly doubled overnight, $12.30 to $24.00. That is above the $20 target (AMBER by rule) and under the $30 escalation line ledger row 59 set. Reads per verified skill hold at 5, and 99 billed reads against 278 posts kept means no discard gap. The daily budget is fine: 5% used, x_over_budget false.

Overnight

Production last deployed 2026-09-02T01:31:45Z, version 653b02f4, SHA 1f5dcb9. HEAD is 4fcf0d9, so four commits are undeployed: the fix(registry): retry a transient ungh tree read before answering 503 #119 ungh tree-retry fix, the fix(copy): align site copy with v3 run-first positioning #112 copy alignment, the chore(digest): pause the AI summary behind an explicit flag #121 digest summary pause, and chore(routines): schedule the daily check-in #125 (this routine's schedule). No deploy ran for any of them.
The CI probe failed with HTTP 401: Bad credentials, so no workflow state was collected this window. The prior archive read Test green and Deploy to Cloudflare state missing. Whatever stopped the deploy pipeline on 09-02 is invisible until the credential is fixed.
Five commits landed since the last run.

Broken

Deploy freeze since 2026-09-02T01:31Z (new)

deploy.latest.approxDeployedSha is 1f5dcb9, four commits behind HEAD 4fcf0d9, and the newest Cloudflare deployment is 44 hours old. The frozen set includes #119, the retry fix for the skills-raw 503 family that is still firing (SKILLD-1E below), and #121, the pause for the degraded digest summary.

Hypothesis: the deploy trigger stopped after the early 09-02 deploy. Without the CI probe there is no evidence for which half failed: Test red on main, the deploy workflow, or the runner pool. Not the resolved 2026-09-01 runner row by itself; that was a missing label and a later run claimed and succeeded.

`ci probe 401 Bad credentials` (new)

ci.error: couldn't fetch workflows for skilld-dev/skilld.dev: HTTP 401: Bad credentials. The prior archive collected full workflow state, so this is a step change in the routine's own observability. Missing data cannot support GREEN, and every CI gate is an explicit unknown this run.

Hypothesis: the gh credential this routine uses expired or was revoked. Same failure class as the resolved 2026-07-25 Sentry probe 403.

SKILLD-1E grew on a new culprit while its fix sits undeployed (regressed)

SKILLD-1E, Skill source is unavailable upstream, grew 2 to 16 events, last seen 2026-09-03T17:50:54Z, new culprit GET /api/skills-raw/neondatabase/agent-skills/neon-functions. The endpoint answers 200 this morning because ungh is currently healthy; the intermittent 503s are what the id counts. The #119 tree-read retry fixing exactly this is one of the four frozen commits.

Hypothesis: transient ungh blips keep converting to 503s because the retry fix never deployed. The deploy freeze is the cause of the growth, not a new mechanism.

2 user-impacting repos newly broken (new, verified benign)

newly_broken_repos_impacted moved 0 to 2, newly_broken_repos_total 4 to 13. A read-only production query names them: levi-qiao/skillmill (broken 09-02T15:01Z, 1 skill) and aamarva/aamarva-adk (broken 09-02T20:02Z, 1 skill). The 3 skill sync failures are the same story: levi-qiao/skillmill/mini-docs repo_missing, plus cat-xierluo/legal-skills industry-research-report and weekly-legal-briefing path_missing on a still-healthy repository.

Hypothesis: ordinary upstream deletions, and the sync recorded its designed verdicts. Verified read-only this morning: all three skills answer 410 on skills-raw with the tombstone machinery from #100/#109, and dimillian/skills/app-store-changelog answers 200. This matches the nightly health email's amber reasons. Cumulative broken_repos 2,095 is known inventory, not a failure.

D1 overload burst recurred on new ids (known family, ledger rows 45 and 105)

Twelve new ids, all D1_ERROR: D1 DB is overloaded. Requests queued for too long., 233 events confined to 2026-09-02T09:17:07Z through 09:17:12Z. Largest: SKILLD-1F with 195 on GET /api/skill-related/github/awesome-copilot/aws-resource-health-diagnose. One user per id and a five-second window: the crawler-sweep signature. SKILLD-1V added one KV PUT failed: 429 at 10:04:19Z, which is the guarded write path reporting rather than failing the request. workers.nonOk is empty and there is no loadShed this window. Exit condition unchanged: a crawler sweep produces no burst.

Not broken, stated explicitly

https://skilld.dev/ and /skills both 200.
workers.nonOk empty: 122,233 successes, 61 client disconnects.
Migration heads match at 0117_email_cadence_split.sql, no uncommitted migrations, missingExpectedTables empty.
13 failed jobs, all accounted: 11 no_supported_skill_paths are terminal submission rejections (resolved class, ledger row 58), 1 repo fetch 404 is the expected terminal close of a deleted repository (resolved row 89), and 1 blob_batch_failed:502 at 09-03T08:07Z is a single transient GitHub 502. Its batch was the only non-clean one of the last ten, and every later batch finished clean, so it is a recovered single, not an incident.
Health email for 2026-09-03 sent on schedule, AMBER, three reasons: send-digests partial (known; the pause is frozen with everything else), the 2 user-impacting repos, and 1 skill sync failure. All three are accounted for above.
All 18 scheduled tasks report ok or the known send-digests partial. Nothing stale, no stale reserved jobs.

Drift and observability

Deploy SHA: production 1f5dcb9 vs HEAD 4fcf0d9, 44 hours. This is the RED.
CI: probe 401, no workflow state. Observability gap, and the deploy freeze cannot be diagnosed until it is fixed.
X spend: $24.00/mo projected, above the $20 target, below the $30 escalation line in ledger row 59.
D1 overload family recurred on fresh ids; per row 105 the count is not a step change.

Proposed actions

Restore the check-in's GitHub credential. Run gh auth login (or refresh the token the routine uses), then re-run the probe read-only. Nothing CI-related can be judged until this returns data.
Unfreeze deploys and ship the four waiting commits. With CI readable, read Test and Deploy to Cloudflare state on 4fcf0d9, fix whatever skipped, and deploy, so fix(registry): retry a transient ungh tree read before answering 503 #119 (live 503 fix) and chore(digest): pause the AI summary behind an explicit flag #121 (digest pause) reach production. Verify deploy.latest.approxDeployedSha equals main HEAD and SKILLD-1E stops growing.
Watch X spend at the new level, do not act yet. $24.00 is a near-double but under the $30 escalation line row 59 set, and reads per verified skill hold at 5. Escalate only if the next archive passes $30 or the ratio crosses 6.

The check-in CI probe answers HTTP 401 Bad credentials, so no workflow state is collected and every CI gate, including the deploy freeze diagnosis, is unverifiable.

Target: scripts/tools/daily-checkin-data.mjs (GitHub credential used by the ci probe)

Verify with: node scripts/tools/daily-checkin-data.mjs exits with no ci.error and a populated ci.workflows array

Estimated to change 1 file.

Production is frozen at 1f5dcb9, four commits behind main since 2026-09-02T01:31Z, leaving the #119 skills-raw 503 fix and the #121 digest summary pause undeployed.

Target: .github/workflows/deploy-cloudflare.yml

Verify with: node_modules/.bin/wrangler deployments list --json --config wrangler.jsonc reports the newest approxDeployedSha equal to origin/main HEAD, and curl -s -o /dev/null -w '%{http_code}' https://skilld.dev/api/skills-raw/dimillian/skills/app-store-changelog keeps returning 200 while SKILLD-1E stops growing

Estimated to change 1 file.

Daily check-in 2026-09-04

RED. The deploy gate is red on main and production has been frozen 12 commits behind since 2026-09-02, while the front door itself stayed healthy and the X spend projection crossed the $20 target.

Pulse

Metric	Value	Prior
Users	24	24
Skills	11,908 (+366 new)	11,542 (+62)
Repos changed in window	158	48
Installs	2	6
Digests sent / failed	0 / 0 (monthly cron)	2 / 0
Known AI cost	$0.00 recorded; unmeasured lanes unknown	$0.00
X spend	$29.85/mo projected (117/2000 reads today)	$12.30/mo

This window is ~68 hours (2026-09-02T01:20Z to 2026-09-04T21:44Z). The prior one was ~22 hours. Compare rates, not totals. Skill inflow roughly doubled to ~129/day from ~68/day. Embeddings: 229 completed, 0 failed, 0 stuck. New skill sync failures: 29, up from 0. Newly broken repos: 20 total, 3 impacted (the gate metric), up from 4 and 0. Cumulative broken_repos at 2,101 is known inventory.

X: x_over_budget false, 6% of the daily budget used, 5 reads per verified skill (the escalation line is 6). 422 posts kept in 24 hours against 117 billed reads today, so there is no discard gap. The projection doubled on volume: x_posts_24h moved 144 to 422 and 101 skills verified in 24 hours.

Overnight

No deploy since 2026-09-02T01:31Z. Production is version 653b02f4 at 1f5dcb9. origin/main is 5d2e651, 12 commits ahead, including fix(registry): retry a transient ungh tree read before answering 503 #119 (SKILLD-1E retry fix), chore(digest): pause the AI summary behind an explicit flag #121 (digest AI-summary pause flag), and the 2026-09-04 marketing and CI batch.
Test on the main tip failed at published-cli-grammar (run 33839195925). Every main-branch Test run since 2026-09-02T03:31Z failed or was cancelled, so Deploy to Cloudflare always skipped. One exception matters: 827faaf passed Test at 2026-09-02T02:25Z, but main had already moved, so its deploy skip was the guard working.
The archive's ci probe itself failed: HTTP 401: Bad credentials. Read-only investigation found the environment's GITHUB_TOKEN is invalid while gh's keyring login is valid. gh prefers the env var, so the probe was blind. All CI facts above come from that out-of-band read with the env var unset. This is an observability gap, not a GitHub outage.

Broken

Deploy frozen: the CLI grammar gate cannot read a one-day-old release (new)

pnpm cli:grammar on the main tip reports skilld@beta is 3.0.0-beta.4: could not read its command list; could not read its --agent values. Verified read-only: pnpm dlx skilld@3.0.0-beta.4 --help answers ERR_PNPM_NO_MATURE_MATCHING_VERSION. The runner's pnpm minimumReleaseAge policy (24 hours) refuses any release younger than a day, and skilld@3.0.0-beta.4 was published 2026-09-04T04:18Z. The checker then reads the pnpm error text as help, both parsers return null, and the check fails.

Result: every main Test since 09-02 fails or is cancelled, the gated deploy always skips, and production froze at 1f5dcb9 with 12 undeployed commits. One earlier main Test failure was different: 4fcf0d9 on 09-03 died in build with exit 129 (SIGHUP) despite the heartbeat wrapper. It did not recur on 09-04; treated as a runner flake, no fingerprint.

Hypothesis for the class defect: the gate can never pass on the day a CLI release publishes, so any merge same-day as a release freezes deploys for 24 hours. pnpm-workspace.yaml already carries minimumReleaseAgeExclude entries for published packages this repo depends on (skilld-protocol@2.4.0 among them); skilld itself is missing from that list.

Impact: Loop 1 and Loop 2 both. The backlog holds the SKILLD-1E retry fix, the digest AI-summary pause, and four marketing or SEO fixes.

D1 overload family recurred across the whole window (regressed, known family)

12 new ids, all D1_ERROR: D1 DB is overloaded. Requests queued for too long., about 244 events from 2026-09-02T09:17Z through 2026-09-04T08:23Z. SKILLD-1F carries 202 alone on GET /api/skill-related/dylantarre/animation-principles/universal-emotion; the rest are 1 to 12 events. userCount is 1 on every id, the crawler-traversal signature the 2026-08-05 row named. Unlike the 09-01 four-second sweep, this spread over days. Workers loadShed moved 0 to 13 in the same window. Known family, ledger rows of 2026-08-05 and 2026-09-01; exit condition still unmet: a crawler sweep must pass with no burst.

`SKILLD-1E` grew while its fix sits in the frozen backlog (regressed)

SKILLD-1E grew 2 to 16 events on a new culprit, GET /api/skills-raw/neondatabase/agent-skills/neon-functions, last seen 2026-09-03T17:50:54Z. The #119 tree-read retry fix merged on 09-02 but is undeployed because of the freeze, so growth is expected until the backlog ships. Ledger exit condition (one window with no growth) is unreachable until deploys resume.

`SKILLD-1V` KV PUT 429 escaped after the `writeCache` class fix (new id, known class)

SKILLD-1V, KV PUT failed: 429 Too Many Requests, culprit GET /api/skills/vercel-labs/agent-skills/react-best-practices, 1 event, 1 user, 2026-09-02T10:04:19Z. The event landed on the version deployed 09-02T01:31Z, so after every writeCache guard. Every application setItem routes through writeCache, and a static test enforces it. But the skill detail route is wrapped in defineCachedEventHandler with swr: true (layers/registry/server/api/skills/[...slug].get.ts:456), and Nitro's own route-cache writes hit the KV-backed storage outside that guard. Hypothesis: the SWR revalidation PUT is the escaping write, the one path the SKILLD-C class fix never covered.

Not broken, stated explicitly

https://skilld.dev/ and /skills both 200.
Migration heads match at 0117_email_cadence_split.sql, local and production. No uncommitted migrations. missingExpectedTables is empty.
21 failed_jobs in the window split three ways: 18 no_supported_skill_paths are terminal submission decisions (ledger row of 2026-08-17, excluded from health); 2 repo fetch 404 are the resolved design recording one verbatim row per deleted repository; 1 blob_batch_failed:502 at 2026-09-03T19:26Z is the window's only genuine fault, first equals last, recovered, and it is the "1 job failed" in the 09-04 operator email.
All 10 recent job batches finished failed_jobs: 0, newest registry-sync:1788555664 on 09-04. No active incident.
Health email ran on cadence both nights and was delivered: 09-03 AMBER (send-digests partial, 2 unavailable repos, 1 sync failure) and 09-04 RED (1 job failed, send-digests partial, 2 new skill sync failures). Reasons archived as required.
17 of 18 scheduled tasks ok. send-digests is still partial from the 09-01 monthly run; the pause decision (chore(digest): pause the AI summary behind an explicit flag #121) is undeployed. Next natural run 2026-10-01.
loadShed 13 over ~68 hours against 186,548 successes (0.007%), no internalError. Low rate; watched, no action.
registry_maintenance holds one complete task. No stale reserved jobs, no stuck AI batches.

Drift and observability

Deploy SHA drift: production 1f5dcb9 vs main 5d2e651, 12 commits, 2.8 days. The drift and the red gate are one incident (above).
CI observability gap: the check-in probe answered 401 all window because a stale GITHUB_TOKEN env var shadows gh's valid keyring login. The gate had to be read out-of-band.
skill_sync_failures 0 to 29 and newly_broken_repos_impacted 0 to 3: matches the 09-04 email's sync-failure reason. Watch the next window; 3 impacted repositories back live skills.
X spend $29.85/mo projected against the accepted $20 target, one dollar under the ledger's $30 escalation line. AMBER by rule.
AI cost recorded is $0.00 (ai_submitted 0). The generation lane is paused by flag; unmeasured services stay unknown.

Proposed actions

Unfreeze deploys. Add skilld to minimumReleaseAgeExclude in pnpm-workspace.yaml, so pnpm dlx skilld@<fresh> works on release day and the grammar gate tests the real CLI. Then confirm pnpm cli:grammar is green and run Deploy to Cloudflare by hand to ship the 12-commit backlog.
Repair the check-in CI probe. When GitHub answers 401 and a GITHUB_TOKEN/GH_TOKEN env var is set, retry with the env var stripped so gh falls back to the keyring. One stale env var should not blind the whole gate.
Close the SKILLD-1V escape path. Make Nitro's route-cache write best-effort or route it under the cache guard, starting with the skill detail route, so a KV 429 can never 500 a read again.
Decide the X spend target this window. The projection is $29.85 against the $20 target. The spend is still productive (5 reads per verified skill, no discard gap), so raise the target deliberately or cap daily discovery reads.
After the deploy lands, confirm SKILLD-1E stops growing. One clean window on the id closes the ledger row.

The deploy gate cannot run skilld@3.0.0-beta.4 because pnpm's 24h minimumReleaseAge policy refuses the fresh release, so every main-branch Test fails and production is frozen 12 commits behind.

Target: pnpm-workspace.yaml

Verify with: pnpm dlx skilld@3.0.0-beta.4 --help && pnpm cli:grammar

Estimated to change 1 file.

The check-in CI probe dies on a 401 when the environment carries a stale GITHUB_TOKEN instead of falling back to gh's valid keyring login, blinding the workflow gate.

Target: scripts/tools/daily-checkin-data.mjs

Verify with: GITHUB_TOKEN=invalid node scripts/tools/daily-checkin-data.mjs | jq -e '.ci.workflows'

Estimated to change 2 files.

Nitro's route cache writes KV outside writeCache on the skill detail route, so a KV 429 still escapes as a 500 (Sentry SKILLD-1V).

Target: layers/registry/server/api/skills/[...slug].get.ts

Verify with: pnpm vitest run test/unit/cache-write-best-effort.test.ts

Estimated to change 2 files.

RED: The front door answers 200, but the CI probe lost GitHub auth, production is frozen 12 commits and 4 days behind main with fixes for live 503s waiting, and the D1 overload family burst twice.

Checked 2026-09-06T03:29Z against the 2026-09-02 archive. Window: 2026-09-02T01:20Z to 2026-09-06T03:29Z, about 4 days. The routine did not run on 09-03, 09-04, or 09-05, so no daily baselines exist for those dates. The CI probe failed, so no workflow state is in this archive. Missing data cannot support a lower color.

Pulse

Users: 24 (0 new). Installs: 3 events.
Skills: 11,996, up 454. Repo changes: 233. Repos: 8,325.
Digests: 0 sent in the window. The monthly task last ran 2026-09-01 and recorded partial. 0 failed.
Known AI cost: $0.00 recorded. Generation and the digest summary lane are paused. Unmeasured services are unknown, not $0.
X spend: $17.70/mo projected (21/2000 reads today). x_over_budget is false, 1% of the daily budget used. The figure sits inside the accepted $20 target and below the $30 escalation. 150 skills verified in 24 hours at 5 reads per verified skill. No discard gap: 618 posts survived against 21 billed reads.

Overnight

Deploy: production version 653b02f4 went live 2026-09-02T01:31Z at SHA 1f5dcb9. No deploy since.
Main moved 12 commits ahead of production between 09-02T02:23Z and 09-04T05:05Z. The queue holds: fix(registry): retry a transient ungh tree read before answering 503 #119 (ungh tree retry, the SKILLD-1E fix), chore(digest): pause the AI summary behind an explicit flag #121 (digest AI summary pause), fix(artifact-delivery): use manual redirects; workerd rejects redirect error #115 (artifact delivery manual redirects for a workerd restriction), fix(ci): stabilize main deploy checks #132 (deploy check stabilization), fix(ops): sample deeper when skipped guard runs hide the deploy verdict #118 (deploy verdict sampling), plus 4 features and 3 more fixes.
CI: the probe failed with HTTP 401 Bad credentials. No workflow states collected. First archive with zero CI visibility.
Front door: / and /skills both 200.

Broken

CI probe 401, new fingerprint. ci returned an error instead of workflow states, so the gate cannot be read. Read-only check: gh auth status says the token in GITHUB_TOKEN is invalid, and git fetch origin fails on the same token. Hypothesis: the PAT in the routine environment expired or was rotated; it also masks the valid keyring token.
SKILLD-1E regressed, 2 to 17 events, last seen 2026-09-05T01:14Z on a new culprit, GET /api/skill-files/bitwarden/ai-plugins/auditing-external-claude-plugins (permalink). The fix(registry): retry a transient ungh tree read before answering 503 #119 retry fix is merged but undeployed: production predates it by about an hour. Hypothesis: the single-shot ungh tree read still turns one bad answer into a 503 on the run-command surface.
D1 overload family burst twice, watched. 12 new ids, 243 events, every one D1_ERROR: D1 DB is overloaded. Requests queued for too long. First sweep 2026-09-02T09:17Z, second 2026-09-04T08:23Z. SKILLD-1F carries 202 on GET /api/skill-related/... again. userCount is 1 on every id, the crawler signature. No deploy sits between the sweeps.
SKILLD-1V, new fingerprint. KV PUT failed: 429 Too Many Requests escaped as a 500 on GET /api/skills/vercel-labs/agent-skills/react-best-practices (permalink), 1 event, 2026-09-02T10:04:19Z. Every hand-written cache write routes through writeCache, but skill detail is a defineCachedEventHandler, and Nitro's internal route-cache write bypasses it. Hypothesis: a hot route-cache key hit the one-write-per-second KV limit inside Nitro's own storage path.
Nightly health reasons, new in this window. The 09-05 and 09-06 operator reports name a reviewed leaderboard repository invisible for over 15 minutes, two nights running. The 09-05 report adds 1 user-impacting repository unavailable and 13 skills with new sync failures; 09-06 names 2 skills. newly_broken_repos_impacted moved 0 to 3, which is the gated figure. skill_sync_failures read 32 over the window.
Note, not a row: one blob_batch_failed:502 on 09-03T18:06Z, the superseded chunking fingerprint. Single occurrence, every job batch after it finished clean. Treated as a transient GitHub 502.

Drift

Deploy SHA: production 1f5dcb9 against main 5d2e651. 12 commits, 4 days. This is the drift that keeps findings 2 and the send-digests pause from landing.
Migrations: localHead equals prodHead at 0117_email_cadence_split.sql. No uncommitted migrations. No missing expected tables.
send-digests: sync_jobs still holds partial from the 2026-09-01 monthly run (AI Gateway credits, users 39 and 42). The nightly email carries it forward. chore(digest): pause the AI summary behind an explicit flag #121 would stop the failure at the October run, but is undeployed.
Workers: 13 loadShed against 260,925 successes, 0.005%, daily rates level with the prior window. Verdict: minor, correlated with the crawler sweeps, no internalError. Stays inside the D1 family watch.
Observability gaps: the CI probe is down (finding 1). The routine itself skipped three days. Both hide rather than measure.

Proposed actions

Refresh the invalid GITHUB_TOKEN on the check-in runner, or remove it so gh falls back to the valid keyring token. Restores the CI gate and git fetch in one step.
Deploy main to production. This ships fix(registry): retry a transient ungh tree read before answering 503 #119 (stops the SKILLD-1E 503 family on the run-command surface), chore(digest): pause the AI summary behind an explicit flag #121 (stops the monthly digest AI failure the nightly email re-raises), fix(artifact-delivery): use manual redirects; workerd rejects redirect error #115, fix(ci): stabilize main deploy checks #132, and 12 commits of feature work. If the deploy gate is red, the repaired CI probe will show it.
Make Nitro's route-cache writes best-effort: mount a write-safe wrapper over the cache storage so a KV 429 inside defineCachedEventHandler becomes a miss with a wide event, not a 500. Mirrors writeCache semantics for Nitro's own writes.
Name the reviewed leaderboard repository invisible for over 15 minutes, read-only, and check its skill_repo_eligibility verdict against its skills row count. Two consecutive nightly reports; the 2026-08-04 gate fix covered the last repository with this signature.

The GITHUB_TOKEN on the check-in runner is invalid, so the CI probe returns HTTP 401 and no workflow state reaches the archive.

Target: scripts/tools/daily-checkin-data.mjs (GITHUB_TOKEN in the routine runner environment)

Verify with: gh auth status reports a valid token, then node scripts/tools/daily-checkin-data.mjs archives ci.workflows states

Estimated to change 1 file.

Production sits at 1f5dcb9 from 2026-09-02 while main moved 12 commits ahead, leaving merged fixes for live SKILLD-1E 503s and the digest AI failure undeployed.

Target: .github/workflows/deploy-cloudflare.yml

Verify with: node scripts/tools/daily-checkin-data.mjs records approxDeployedSha equal to main HEAD

Estimated to change 1 file.

The defineCachedEventHandler route cache writes KV directly, so a KV 429 escapes writeCache and 500s the skill detail endpoint.

Target: shared/server/cache.ts

Verify with: pnpm vitest run test/unit/cache-write-best-effort.test.ts

Estimated to change 2 files.

The nightly health check reports a reviewed leaderboard repository invisible for over 15 minutes on two consecutive nights without naming the repository or its gate verdict.

Target: layers/identity/server/utils/daily-health-check.ts

Verify with: pnpm vitest run layers/identity/server/utils/daily-health-check.spec.ts

Estimated to change 2 files.

AMBER: the front door is healthy and the deploy is current, but the CI gate probe 401s, the D1 overload family regressed with 245 events, and the X spend projection crossed the accepted escalation line.

Window: 2026-09-02T01:20:49Z to 2026-09-06T21:43Z (4.8 days), archive 2026-09-06.json.

Pulse

Users 24 (+0). Skills 12,078 (+536). Repos 8,357; 270 changed. Installs 3. Collections 10.
Digests: 0 sent, 0 failed. send-digests is monthly; its last run was 2026-09-01, before this window.
AI cost: $0.00 recorded batch estimate. Unmeasured services unknown, not $0.
X spend: $34.20/mo projected (138/2000 reads today). Projection crossed the $30 escalation line set 2026-08-17. Reads per verified skill: 5. Posts 24h: 740, verified skills 24h: 203. Budget used 7%, so discovery is not truncating. x_over_budget false.
Embeddings: 415 completed, 0 failed, 0 stuck. New sync failures on skills: 33. Newly broken repos: 29 total, 3 impacted (AMBER gate). Broken-repo inventory 2,110, known cumulative.

Overnight

13 commits since the last archive. Deploy 2026-09-06T06:14Z, version 2a783e39, at HEAD 5d2e651. Production is current.
CI gate blind: the workflows probe failed with HTTP 401: Bad credentials. No workflow state was archived, so CI health is unobservable this run. The deploy itself succeeded, which is the only signal; no inference beyond that.
Front door: 200 on / and /skills.

Broken

New or regressed fingerprints only.

D1 overload family regressed (ledger: D1 overload burst on a four-second crawler sweep). 12 new Sentry ids, all D1_ERROR: D1 DB is overloaded, 244 events. Most opened in one burst 2026-09-02T09:17:07Z to 09:17:11Z; events trail to 2026-09-04T08:23Z, so this was not a four-second sweep. SKILLD-1F alone carries 202 events on GET /api/skill-related/dylantarre/animation-principles/universal-emotion (permalink, culprit as shown). Hypothesis: the route caches with a 1-hour TTL, and every expiry during a crawler sweep is a live multi-query recompute into an already loaded D1, so a hot URL re-fails roughly hourly until the sweep passes.
SKILLD-1E grew 2 to 17 events, last seen 2026-09-05T01:14Z (ledger watch). Same id, Skill source is unavailable upstream on live sources. Hypothesis: transient ungh blips still exhaust the tree retry budget and land in the 503 lane.
SKILLD-1V KV PUT failed: 429, 1 event 2026-09-02T10:04Z, culprit GET /api/skills/vercel-labs/agent-skills/react-best-practices (permalink). New id after the writeCache guard deployed. Verdict: watch. Every server cache write routes through writeCache, which swallows and emits a wide event, so this reads as the guard surfacing the rejection rather than a 500; single event, no recurrence in 4.8 days.
X projection $34.20/mo passed the $30 escalation (ledger: X discovery projects over the monthly spend target). Spend stays productive: 5 reads per verified skill and 203 verified in the last day. This is now a decision point, not a fault.
CI probe 401 (new fingerprint). GITHUB_TOKEN in the routine environment is invalid; the keyring account harlan-zw holds a valid token but GITHUB_TOKEN overrides it. The gate cannot report until the credential is fixed.
Nightly report re-alarms on a month-old partial verdict (new fingerprint). send-digests last ran 2026-09-01 and recorded partial (AI gateway credits). chore(digest): pause the AI summary behind an explicit flag #121 paused the summary on 2026-09-02, so the next run, 2026-10-01, will report ok. Until then the nightly email carries Scheduled tasks partially failed: send-digests every night; the 2026-09-05 and 2026-09-06 reports both do. The cause is decided; the alarm is stale.
33 skill sync failures and 3 impacted newly broken repos in the window. Health emails name 13 new failures (09-05) and 2 (09-06), plus 1 user-impacting repo unavailable (09-05) and 1 reviewed leaderboard repo invisible over 15 minutes on both nights. The archive records counts, not identities, so repair versus prune is not decidable from the report.

Verdicts on recovered or bounded signals

Workers loadShed 13 against 324,107 successes (0.004%). Bounded, no trend evidence. Note only.
repo fetch 404 (2) and blob_batch_failed:502 (1): last failures 2026-09-04 and 09-03; every recent registry-sync batch finished with 0 failed jobs. Recovered transients.
no_supported_skill_paths (28): terminal submission decisions, per the resolved 2026-08-17 ledger class. Not faults.
Batch registry-sync:1788724825 shows 1 job still pending, last updated ~4.5 hours before the probe, while the next hour's batch finished clean. Watch note; becomes a fingerprint if it persists.
daily-health-check last ran 2026-09-05T22:00Z, ok, on cadence for tonight's 22:00Z run.

Drift

Migrations: local head 0117_email_cadence_split.sql, prod head identical. No drift, none uncommitted.
Health email: one sent AMBER report per Melbourne date (09-05, 09-06). Reasons reported above, per gate.
Deploy SHA equals HEAD. No stale scheduled task, no stale reserved job, no missing table.
Observability gaps: CI workflows unreadable this run (401); impacted broken repos archived as counts without identities.

Proposed actions

Refresh or remove the invalid GITHUB_TOKEN in the routine environment so the CI gate probe reads workflows again. The keyring token already works; clearing the env var restores the gate with no code change.
Serve stale cache when a cached read's recompute fails, starting with /api/skill-related. A TTL expiry during a D1 overload currently costs the user a 500; a stale copy costs nothing. Extends shared/server/cache.ts alongside readCache/writeCache.
Bound the scheduled-task partial alarm to the task's current cron period in daily-health-check.ts, so a monthly task's old verdict stops going nightly AMBER while its fix waits for the next natural run.
Decide the X discovery spend: projection $34.20/mo passed the $30 escalation line accepted 2026-08-17. Tune reads per verified skill or raise the target deliberately; both are one-line decisions.
Archive the identities of impacted newly broken repos in the daily JSON, so the 3-repo AMBER gate names what to repair or prune instead of a count.

The environment GITHUB_TOKEN is invalid, so the workflows probe answers 401 and no CI state is archived while the gh keyring token works.

Target: routine environment GITHUB_TOKEN used by scripts/tools/daily-checkin-data.mjs CI probe

Verify with: node scripts/tools/daily-checkin-data.mjs --save && jq -e '.ci.workflows' docs/ops/checkins/$(date -u +%F).json

Estimated to change 1 file.

When a TTL'd cache entry expires during a D1 overload, the live recompute rejection 500s the request instead of serving the last good response, as SKILLD-1F's 202 events show.

Target: shared/server/cache.ts plus layers/registry/server/api/skill-related/[...slug].get.ts

Verify with: pnpm vitest run test/unit

Estimated to change 3 files.

The scheduled-task alarm fires on any partial last_status with no time bound, so send-digests' 2026-09-01 partial verdict drives nightly AMBER until its 2026-10-01 run even though #121 paused the cause.

Target: layers/identity/server/utils/daily-health-check.ts

Verify with: pnpm vitest run layers/identity/server/utils/daily-health-check.spec.ts

Estimated to change 2 files.

The projected monthly spend reached $34.20, crossing the $30 escalation line accepted on 2026-08-17, so the target must be raised deliberately or discovery tuned down.

Target: X discovery budget configuration (x_budget_target)

Verify with: node scripts/tools/daily-checkin-data.mjs | jq '.d1.cost.x_projected_monthly_usd'

Estimated to change 1 file.

The archive records only counts for impacted newly broken repos, so the 3-repo AMBER gate names nothing to repair or prune.

Target: scripts/tools/daily-checkin-data.mjs

Verify with: node scripts/tools/daily-checkin-data.mjs | jq '.d1.pipeline'

Estimated to change 1 file.

RED: `Test` is red on `main` and deploys are frozen behind HEAD, and X discovery now projects $42.60/mo past the $40 accepted line.

Window note: the prior archived run was 2026-09-02, so this window covers 5d 20h (2026-09-02T01:20Z → 2026-09-07T21:40Z), not one night. Nothing archived for 9/3–9/6; see Drift.

Pulse

Users 24 (0 new). Skills 12,247 (+705, was +62). Repos 8,400 (+216). Owners 9,807. Collections 10.
Repo changes 309, installs 7, digests 0 sent / 0 failed (monthly cadence; last ran 2026-09-01), health email sent both nights.
Known AI cost: $0.00 recorded (ai_batch_costs; generation paused, so unmeasured services are unknown, not $0).
X spend: $42.60/mo projected (166/2000 reads today), 915 posts and 262 skills verified in 24h, 4 reads per verified skill, no discard gap, not falling behind.
Front door: 200 on / and /skills.

Overnight: deploy and CI

Deploy to Cloudflare: success 06:15Z on 8b1af9f (version 2e84908c); every run since is skipped.
Test: failure on HEAD b479402 (run 34139271656), second consecutive failure. Two repair runs queued at 21:02Z and 21:05Z (9a1fcb9 sentry-capture fix, 66a1efa unused-import fix) and must be followed to completion.
29 commits since the last archived run, including the taste-tested homepage (feat: taste-tested homepage, tracks ordered by measured demand #136) and the run-command → Skill-page-URL change (feat(registry): hand agents the Skill page URL, not the run command #172) that broke the gate.

Broken

Test gate red on main after feat(registry): hand agents the Skill page URL, not the run command #172 (new fingerprint, watch). Run 34139271656 failed on b479402; Deploy is gated on Test, so production is frozen at 8b1af9f (06:15Z) while main moved to b479402 plus two queued fixes. Hypothesis: feat(registry): hand agents the Skill page URL, not the run command #172 left an unused skillRunCmd import; the queued 66a1efa removes it. Exit: Test green on main, Deploy lands that SHA.
3 newly broken repos are impacting (gated signal, was 0). iamzhihuix/happy-claude-skills, aamarva/aamarva-adk, levi-qiao/skillmill each still back skills. Cumulative broken inventory 2,126 is known and not a failure; the 3 impacted identities are the finding.
SKILLD-1E grew 2 → 17 events (existing watch). "Skill source is unavailable upstream", last seen 2026-09-05T01:14Z, after the fix(registry): retry a transient ungh tree read before answering 503 #119 retry fix deployed. Per its own exit condition, further growth on live sources means the outage classification over-fires; the queued 9a1fcb9 drops expected upstream-outage 503s from capture, so the next window reads cleanly either way.
D1 overload burst, recovered (existing family, rows 45/105). 13 new ids, 242 events, every event last seen by 2026-09-04T08:23Z; SKILLD-1F carries 202 on one skill-related culprit. Same crawler-sweep signature (userCount 1 everywhere), no event in ~3 days. Plus SKILLD-1V KV PUT 429 ×1 (9/2), covered by the fix(cache): resolve route cache KV write failures as wide events #159/fix(cache): serve stale cached responses when a recompute fails #168 cache fixes deployed 9/7.
blob_batch_failed:502 recurred once (old fingerprint). Single row, first = last 2026-09-03. Recovered burst, no action.
37 skill sync failures in window (was 0), no identity archived. The 9/6 nightly named 2 in 24h. Hypothesis: upstream churn across the 5-day window. Cannot triage without identities; needs the same naming treatment the invisible-review reason got.

Drift

Production ref probe failed (new fingerprint, watch). git fetch origin main dies with "Invalid username or token": the stale GITHUB_TOKEN env is what git's gh auth git-credential helper serves, and fix(daily-checkin): let gh fall back to its keyring when the token is stale #133's keyring fallback scrubs the env for gh commands only. Two consequences in the archive: approxDeployedSha is null and d1.migrations is an error, so deploy-SHA match and migration drift are unverified this window. Fix mirrors fix(daily-checkin): let gh fall back to its keyring when the token is stale #133: apply ghEnv to the collector's git invocations.
No archived check-ins 9/3–9/6. The prior baseline was 2026-09-02; five days had no durable production record and no artifact to diagnose the misses from.
Workers loadShed 13 against 417,621 successes (0.003%), no internalError. Verdict: bounded, watch only.
Health email sent both nights, AMBER. 9/7 report: "Scheduled tasks partially failed: send-digests." + "1 reviewed leaderboard repository remained invisible for over 15 minutes." 9/6 report: the same two plus "2 skills recorded a new sync failure in 24 hours." Both reports predate fix(ops): name invisible leaderboard repos in nightly report #149's identity-naming fix (worker 2a783e39 deployed 06:14Z 9/7), so the unnamed repo is expected there; tonight's report is the fix's first natural test.
send-digests partial is known state: monthly cron, AI summary paused since chore(digest): pause the AI summary behind an explicit flag #121, last run 2026-09-01; the partial status shows until the next natural run.
x_ledger_held 134 → 150 (+16 in window). Post-fix(admin): accept session auth for admin endpoints #62 these should be person-actionable holds; rate noted, no verdict yet.

Proposed actions

Follow the two queued Test runs (34161857259 on 66a1efa, 34161669940 on 9a1fcb9) to green and confirm Deploy lands main; production has been frozen at 8b1af9f since 15:37Z while fixes wait. (follow-up)
Scrub the stale token env for the collector's git invocations so the production ref and migration drift probes resolve, mirroring fix(daily-checkin): let gh fall back to its keyring when the token is stale #133's ghEnv fallback. (PR)
Decide the X escalation at $42.60/mo: cap DAILY_DISCOVERY_READ_BUDGET at 250 reads/day, which holds the projection at $37.50/mo and makes daily truncation the designed steady state, or raise the accepted line again. (PR or decision)
Review the 3 impacted newly broken repos (iamzhihuix/happy-claude-skills, aamarva/aamarva-adk, levi-qiao/skillmill) and re-source or tombstone each. (ops)
Check tonight's operator report names the invisible reviewed repository; it is the first natural run on a worker with fix(ops): name invisible leaderboard repos in nightly report #149. (verification)

The collector runs git fetch origin main with the stale GITHUB_TOKEN env still set, so git's gh auth git-credential helper serves that invalid token instead of the valid keyring login and git.productionRef plus d1.migrations both error.

Target: scripts/tools/daily-checkin-data.mjs

Verify with: node scripts/tools/daily-checkin-data.mjs | node -e "let s='';process.stdin.on('data',d=>s+=d).on('end',()=>{const j=JSON.parse(s);if(j.git.productionRef.error||!j.d1.migrations.localHead)process.exit(1)})"

Estimated to change 2 files.

X discovery projects $42.60/mo, over the $40 accepted line, because the 2000-read DAILY_DISCOVERY_READ_BUDGET never binds at the observed 284 reads/day and needs lowering to about 250.

Target: shared/server/x-ingest.ts

Verify with: pnpm vitest run test/unit/x-ingest.test.ts

Estimated to change 1 file.

RED: X discovery projects $76.20/mo past the $60 red line, the D1 overload family threw again after the #158/#168 fixes deployed, and 3 newly broken repositories still back live skills.

Window: 2026-09-02T01:20Z → 2026-09-08T21:42Z (~6.5 days, vs a 1-day prior window; per-day rates noted where it matters).

Pulse

Users: 24 (0 new), installs 10.
Skills: 12,422 (+880, ~135/day vs 62 the prior day). Repo changes: 353.
Digests: 0 sent, 0 failed (monthly cadence; next natural run Oct 1).
Known AI cost: $0.00 recorded. Generation and digest summary are paused; unmeasured services are unknown, not $0.
X spend: $76.20/mo projected (283/2000 reads today). 4 reads per verified skill; 420 skills verified in 24h (was 22). Posts 24h 1,213 vs 283 billed reads today, so no discard gap.
Inventory: repos 8,454, owners 9,823, cumulative broken_repos 2,143 (known inventory, not an overnight event).

Overnight

Deploy: production moved b6c4caf7 (2026-09-01) → 103991d0, deployed 2026-09-08T07:41Z, 32 commits. Includes fix(d1): retry replayable reads on D1 overload #158 (D1 overload read retry), fix(cache): resolve route cache KV write failures as wide events #159 (KV write wide events), fix(cache): serve stale cached responses when a recompute fails #168 (serve stale on failed recompute), fix(sentry): drop expected upstream-outage 503s from capture #176 (drop upstream-outage 503s from Sentry capture), feat(registry): hand agents the Skill page URL, not the run command #172 (registry hands agents the Skill page URL), homepage rework.
CI: Test passed on HEAD (run 34199659160), Deploy to Cloudflare success (run 34200192698). The newest Deploy run is skipped (guard on a non-main push, expected). Several PR-branch Test runs queued or cancelled 21:00–21:33Z; in-flight, followed next window.
Front door: / and /skills both 200.

Broken

D1 overload recurred after the fix deploy. SKILLD-1F 295 events, culprit GET /api/skill-related/thedivergentai/gd-agentic-skills/godot-genre-rts, lastSeen 2026-09-08T13:05:34Z. New ids SKILLD-1Y (2 events, 13:05:31Z), SKILLD-1X and SKILLD-1W (1 each, 10:54Z). Both 09-08 bursts post-date the 07:41Z deploy carrying fix(d1): retry replayable reads on D1 overload #158 (replayable-read retry) and fix(cache): serve stale cached responses when a recompute fails #168 (stale-serve). Hypothesis: crawler traversal on cold cache keys still pays the D1 read; retries are bounded and a cold key has no stale envelope to serve.
Three newly broken repositories still back live skills. newly_broken_repos_impacted 0 → 3: iamzhihuix/happy-claude-skills, aamarva/aamarva-adk, levi-qiao/skillmill (all reason: skill). skill_sync_failures 0 → 40; newly_broken_repos_total 4 → 64 over ~6.5 days (~10/day vs 4/day). The 09-08 operator email named iamzhihuix/happy-claude-skills invisible >15 minutes and 1 new skill sync failure in its 24h window. Hypothesis: sources deleted or moved upstream and the sync keeps failing before recording the source_resolved = 0 verdict.
X discovery spend: x_projected_monthly_usd $12.30 → $76.20, past the $40 accepted line (raised 2026-09-07, X discovery projection passed the accepted spend escalation #160) and past the $60 red gate. Productive at 4 reads per verified skill (under the 6 escalation), x_over_budget false, x_budget_used_pct 14 — the daily budget holds; the projection does not.
SKILLD-1E grew 2 → 17 events through 2026-09-05T01:14Z on a new culprit (GET /api/skill-files/bitwarden/ai-plugins/auditing-external-claude-plugins). All events predate fix(sentry): drop expected upstream-outage 503s from capture #176 (deployed 09-08), which stops capturing these 503s. Watch next window before treating the classification as over-firing again.
Workers loadShed 0 → 13 against 536,326 successes (~0.002%). Bounded, consistent with the crawler sweeps; no action, watch.
blob_batch_failed:502 recorded once (2026-09-03), single event; every archived registry batch since finished clean. Recovered burst, not an active incident.

Drift

Migrations: local and prod heads both 0117_email_cadence_split.sql. No drift, no uncommitted migrations.
Deploy SHA equals origin/main HEAD (103991d0). No deploy drift.
No stale reserved jobs, no failed digest, no missing expected tables.
send-digests carries the stale 09-01 partial (AI Gateway credits); summary is paused behind DIGEST_AI_SUMMARY_PAUSED since chore(digest): pause the AI summary behind an explicit flag #121. Next natural run Oct 1.
Operator email: sent for Melbourne dates 09-07 and 09-08, both AMBER. 09-08 reasons: "1 skill recorded a new sync failure in 24 hours."; "1 reviewed leaderboard repository remained invisible for over 15 minutes: iamzhihuix/happy-claude-skills." 09-07 reasons: "Scheduled tasks partially failed: send-digests."; "1 reviewed leaderboard repository remained invisible for over 15 minutes."

Proposed actions

Attribute the post-fix(d1): retry replayable reads on D1 overload #158 D1 overload throws: name which read path still surfaces the 500 on a cold-key crawler sweep (replayable classification or cache coverage), then extend retry or stale-serve there. Proves out when a crawler-sweep-scale window records no new D1 overload events.
Verify the 3 impacted newly broken repositories read-only and confirm their skills answer the 410 tombstone; if the sync fails before writing source_resolved = 0, fix the marking path in sync-repo.ts. Proves out when newly_broken_repos_impacted returns to 0.
Decide the X discovery spend line and enforce it: cut DAILY_DISCOVERY_READ_BUDGET in shared/server/x-ingest.ts to ~250 reads/day (holds ~$40/mo) or re-raise the accepted line deliberately. Proves out when x_projected_monthly_usd sits at or under the chosen line.
No-change watches for next window: SKILLD-1E growth after fix(sentry): drop expected upstream-outage 503s from capture #176, loadShed, and the in-flight PR Test runs (org/tags/clusters/users SWR caching) that target the same D1 overload family.

Cold-key crawler sweeps still turn D1 overload into 500s after the #158 retry and #168 stale-serve fixes deployed.

Target: server/utils/db.ts

Verify with: pnpm test:run test/unit/platform-d1.test.ts

Estimated to change 1 file.

Three upstream-broken repositories still back synced skills because the sync fails before recording source_resolved = 0.

Target: layers/registry/server/utils/sync-repo.ts

Verify with: pnpm test:run test/unit/sync-repo-correctness.test.ts

Estimated to change 2 files.

Daily check-in 2026-09-09

AMBER. The seven-day-dark window is clean on the front door, CI, migrations, jobs and scheduled tasks, but the X discovery projection crossed the accepted $40 line at $43.80/month, the D1 overload crawler family recurred on about 20 new ids, and 3 newly broken sources still back skills.

Window note: the previous archive is 2026-09-02, so this window spans 7.85 days and the routine produced no archives from 2026-09-03 to 2026-09-08. Every rate below is per-day against that span; totals are not comparable with the prior one-day window.

Pulse

Metric	Value	Prior	Rate over 7.85d
Users	24	24	0 new
Skills	12,532 (+990 new)	11,542	~126/day vs 62/day
Repos changed in window	392	48	~50/day vs 48/day
Installs	10	6	~1.3/day vs 6/day
Digests sent / failed	0 / 0	2 / 0	monthly cadence; Sep 1 run fell outside this window
Known AI cost	$0.00 recorded	$0.00	generation lanes paused by flag; unmeasured services unknown
X spend	$43.80/mo projected (187/2000 reads today)	$12.30/mo	over the accepted $40 line

Embeddings completed 761, none failed or stuck (~98/day, under the 480/day cap). skill_sync_failures 48 in the window, matching the upstream breakage below. No stale dirty skills, no stale reserved jobs. X detail: x_over_budget false, x_budget_used_pct 9, 4 reads per verified skill (inside the 6 ceiling), 1,421 posts kept against 187 billed reads today so there is no discard gap, and 455 skills verified in 24 hours, so the spend is still buying inventory. x_ledger_held moved 134 to 167, all person-actionable holds since the repo-gone fix.

Overnight

Production moved to 103991d (feat(marketing): ask what a Skill is for before the package steps, feat(marketing): ask what a Skill is for before the package steps #181), deployed as version d1869c1d at 2026-09-08T07:40:37Z. approxDeployedSha equals HEAD, so there is no deploy lag. 33 commits landed since the last check-in, including the digest AI-summary pause (chore(digest): pause the AI summary behind an explicit flag #121) and the upstream-outage 503 capture fix (fix(sentry): drop expected upstream-outage 503s from capture #176).
Deploy to Cloudflare reports state success read through to run 34200192698 on 103991d; the five newer runs are skipped guard runs from branch pushes.
Test reports state success, newest run 34283516010 at 2026-09-09T03:20:02Z on branch SHA 9870775. One branch run failed on 2895e64 (serve last-good raw body during upstream outages) at 2026-09-08T21:32:57Z; neither SHA is on main, so this is branch noise, not a main-line gate failure.

Broken

D1 overload family recurred across the dark week (regressed, ledger rows 45 and 105)

The archive carries ~20 new ids, all D1_ERROR: D1 DB is overloaded. Requests queued for too long., every one userCount: 1, the crawler-traversal signature. Biggest: SKILLD-1F GET /api/skill-related/nvidia/skills/deepstream-sop, 394 events, first seen 2026-09-02T09:17:07Z, last seen 2026-09-09T02:11:48Z; then SKILLD-1M GET /api/orgs/jonathanxdr (17) and SKILLD-1H GET /api/tags/ctest?view=data (13). Newest id SKILLD-23 fired once on GET /api/orgs/kilo-org at 2026-09-09T02:11:47Z. recurringIssues is empty and the probe was not truncated; the family reopens fresh ids per culprit, so the count is spread, not a step change.

Hypothesis: the same mechanism as the 2026-08-05 and 2026-08-31 rows, crawler sweeps saturating D1 queues while read-through caches hold most traffic. The 394 events on a whole-route-cached skill-related path are cold reads racing the sweep, not a cache regression; exit condition for rows 45/105 stays "a crawler sweep produces no burst", and it failed again.

X discovery projection crossed the accepted $40 line (gate breach, ledger row 59)

x_projected_monthly_usd is $43.80 against the $40 line Harlan accepted on 2026-09-07 (issue #160). Per the gate this is AMBER, not RED (RED starts above $60). The spend is productive: 4 reads per verified skill, inside the 6 ceiling that would mean the spend stopped buying skills, and x_over_budget is false with 9% of the daily budget used. The projection is a rate problem, not a day-truncation problem.

Three newly broken sources still back skills (impacting gate, new identities)

newly_broken_repos_total is 69 (cumulative-window count, not overnight) and newly_broken_repos_impacted is 3, each archived with its reason skill: iamzhihuix/happy-claude-skills, aamarva/aamarva-adk, levi-qiao/skillmill. The first also drives the nightly health email's "reviewed leaderboard repository remained invisible for over 15 minutes" reason on both 2026-09-08 and 2026-09-09, and the "1 skill recorded a new sync failure in 24 hours" reason.

Hypothesis: upstream deletions or rewrites; the sync recorded the verdicts and the 410 tombstone path already covers skills whose source is gone. Neither nightly reason is a registry defect; both are honest alarms about upstream state.

The check-in itself was dark for six days (observability, new)

No archives exist between 2026-09-02T01:20:49Z and this run, while .github/routines.yml schedules daily-checkin at 07:40 Sydney daily. The gap produced no signal anywhere in this repository: the archive carries the wide since but nothing flags a stale baseline, and the nightly operator email is a separate Worker task that kept running. Scheduler-side cause (harlan-github-agent) cannot be read from this repo.

Not broken, stated explicitly

https://skilld.dev/ and /skills both 200.
Migration heads match at 0117_email_cadence_split.sql, local and production. No drift, no uncommitted migrations. missingExpectedTables is empty.
71 failed jobs in the window, all three fingerprints accounted for: 68 no_supported_skill_paths (2026-09-02T03:01Z to 2026-09-09T17:15Z) are the resolved terminal-rejection class of ledger row 58; 2 repo fetch 404 on registry/repo-maintenance (2026-09-02T15:01Z to 2026-09-04T02:02Z) are the resolved class of ledger row 89, recorded verbatim and excluded from the nightly email, which named no job failures; 1 blob_batch_failed:502 (single event, 2026-09-03T18:06Z) is a transient upstream 502.
All ten recent job batches finished failed_jobs: 0, newest registry-sync:1788987613 (24 jobs) at 2026-09-09T21:01:26Z. Recovered burst, not an active incident.
All 18 scheduled tasks report ok and fresh. daily-health-check last ran 2026-09-08T22:00:53Z, inside its 36-hour window; the */5 ticks are current to 2026-09-09T21:40:18Z; detect-star-surges ran on schedule at 2026-09-09T04:30:17Z. send-digests still shows last_status: partial, but that row is the 2026-09-01 run before chore(digest): pause the AI summary behind an explicit flag #121 paused the AI summary; the pause is deployed in d1869c1d, and the next monthly run (2026-10-01) is the test.
Workers outcomes: 618,336 successes, 591 client disconnects, 13 loadShed (0.002%, no attribution data in the archive). Bounded noise, watch the next archive for a rate change.
registry_maintenance holds one complete task, nested-assets-v1. Nothing stalled.
Health email: both archived dates (2026-09-08, 2026-09-09) are sent with AMBER reasons reported above. The probe reads LIMIT 2, so per-date coverage across the dark week is unverifiable from the archive (gap below).
Cumulative broken_repos at 2,148 is known inventory, not an overnight event.

Drift and observability

No migration, schema, or deploy-SHA drift.
The probe's healthEmail LIMIT 2 can never prove the "one sent report per Melbourne date" gate beyond two days, which is exactly the span the dark week made blind.
The check-in's own absence carried no flag: a stale baseline is invisible in the archive, so a future reader cannot distinguish a quiet week from a dead routine.
loadShed carries no request-path attribution (known Workers telemetry limit).
AI cost is the recorded batch estimate only; $0.00 means nothing was billed while the lanes are paused, not that services are free.

Proposed actions

Decide the X discovery line. The projection is $43.80/month against the accepted $40 (issue X discovery projection passed the accepted spend escalation #160). Either raise the line deliberately or tune the query; the data does not force tuning, since 4 reads per verified skill is inside the 6 ceiling and 455 skills were verified in 24 hours.
Keep the D1 overload family on watch, name the consumers first. Before any code change, read wrangler d1 insights for the window and check whether SKILLD-1F's culprit GET /api/skill-related/nvidia/skills/deepstream-sop is a cold-cache race or an uncacheable path. The exit condition is unchanged: a crawler sweep with no burst.
Verify the 3 impacted sources. Confirm iamzhihuix/happy-claude-skills, aamarva/aamarva-adk, and levi-qiao/skillmill skills answer the 410 tombstone, then decide removal versus waiting on each.
Close the check-in darkness. Name why the routine skipped 2026-09-03 to 2026-09-08 in the scheduler, and make the collector flag a stale baseline so a silent gap is a visible archive field rather than a surprise window.
Watch send-digests on its 2026-10-01 run. Confirm the paused-summary path records ok instead of partial.

The archive records the since timestamp but nothing derives a baseline gap, so six skipped days read as an ordinary wide window.

Target: scripts/tools/daily-checkin-data.mjs

Verify with: pnpm test:run

Estimated to change 2 files.

The healthEmail query is LIMIT 2, so the one-sent-report-per-Melbourne-date gate cannot be verified beyond the two newest days.

Target: scripts/tools/daily-checkin-data.mjs (healthEmail query)

Verify with: pnpm test:run

Estimated to change 2 files.

AMBER: front door, deploys, and migrations are healthy, but D1 overload 500s recur on hot skill reads, three newly broken repos still back live skills, and the archived CI gate reads PR branch failures as main failures.

Check-in window: 2026-09-02T01:20:49Z to 2026-09-10T21:45:11Z (~8.9 days). The previous archive is 2026-09-02; no check-in ran between 2026-09-03 and 2026-09-09, so this report compares one archive against another across a 7-day routine gap (tracked as issue #195, PR #196 in flight and red).

Pulse

Users 25 (+1). Skills 12,650 (+1,108). Repos 8,523 (+339). Owners 9,836 (+100). Collections 10.
Installs 18 in window (prior archive: 6 in ~22h). New skills +1,108 and changed repos +430 reflect the 8.9-day window, not a 1-day step.
Digests 0 sent, 0 failed. send-digests is a monthly cron; its last run (2026-09-01, partial) is the known paused-AI-summary decision (chore(digest): pause the AI summary behind an explicit flag #121).
Known AI cost: $0.00 recorded batch estimate. Unmeasured services are unknown, not $0.
X spend: $30.30/mo projected (123/2,000 reads today). Under the $40 accepted line (issue X discovery projects past the accepted $40 monthly spend line #179 premise does not hold today). 4 reads per verified skill (accepted ceiling 6). 1,550 posts in 24h against 123 billed reads, so no paid-for-discard gap. x_over_budget false, x_budget_used_pct 6, x_falling_behind false. 981 verified skills (+485 in 24h).
Health email: one sent operator report per Melbourne date for both dates the probe reads (09-09, 09-10), both AMBER. The probe verifies only two dates (Health email probe verifies only two Melbourne dates #194, PR fix(daily-checkin): verify a week of health email reports, not two #197 in flight and red), so 09-03 through 09-08 are unverifiable from the archive.

Overnight (window, not one night)

Deploy: production version 266356ed at 103991d0, equal to origin/main HEAD, deployed 2026-09-08T07:40Z. Eight commits shipped in the window, latest feat(marketing): ask what a Skill is for before the package steps #181.
Deploy to Cloudflare: state success, read through the skipped guard runs. Healthy.
Test: archived state failure with consecutiveFailures 4. Verified read-only: all four verdict-carrying runs are PR branches (fix/issue-195 twice, fix/issue-194, plus a cancelled pair), and the latest main Test run (34199659160) passed on the deployed SHA 103991d0. The main gate is green; the archive mislabels it (new fingerprint below).
Nightly health reasons: 09-10 report names iamzhihuix/happy-claude-skills invisible >15 min; 09-09 report names the same repo plus 1 new skill sync failure. Both AMBER, both sent.

Broken (new or regressed fingerprints only)

CI workflow state reads non-main branch runs (new). The collector's gh run list call passes no --branch, so the archived Test state is the latest run of any branch. Four PR-branch failures archived as a broken gate while main passed on the deployed SHA. Hypothesis: scoping the run list to the production ref makes the false failure unrepresentable.
D1 overload 500s recur (D1 overload 500s recur after the replayable-read retry fix #192, ledger family row 2026-09-01, escalated by evidence). 21 new Sentry ids, all D1_ERROR: D1 DB is overloaded, firstSeen 2026-09-02T09:17Z through 2026-09-10T01:23Z, lastSeen 2026-09-10T15:36Z (hours before this run). Top culprit SKILLD-1F carries 1,030 events on GET /api/skill-files/davila7/claude-code-templates/data-engineer. userCount 1 on every id, the crawler-traversal signature. Workers loadShed 13 in the same window. Hypothesis: the fix(d1): retry replayable reads on D1 overload #158 retry and fix(cache): serve stale cached responses when a recompute fails #168 stale-serve fixes do not cover the cold-key paths these sweeps hit.
Newly broken repositories still back live skills (Newly broken repositories still back live registry skills #193). newly_broken_repos_impacted 3, all reason skill: iamzhihuix/happy-claude-skills, aamarva/aamarva-adk, levi-qiao/skillmill. newly_broken_repos_total 75 is window-volume against cumulative broken inventory 2,154 (+72), not an overnight incident. Hypothesis (from Newly broken repositories still back live registry skills #193): the sync fails before recording source_resolved = 0.
Workers clientDisconnected step change (new watch). 610 in ~8.9 days (~69/day) against 22 in ~22h (~22/day) in the prior archive. Correlated in time with the D1 overload family and its 13 loadShed but not attributed; no code change yet.

Drift

Migrations: localHead and prodHead both 0117_email_cadence_split.sql, no uncommitted migrations. No drift.
Deploy SHA equals HEAD. No drift.
No stale reserved jobs, no stale scheduled tasks, no missing expected tables. All recent job batches finished 0 failed; the failed-job window is a recovered burst: 75 no_supported_skill_paths are terminal submission rejections (decisions, not faults), 2 repo fetch 404 (last 2026-09-04, terminal by design), 1 blob_batch_failed:502 (2026-09-03, transient).
skill_sync_failures 48 in window (prior 0). The nightly probe counted 1 new in its 24h. Likely the same repos as Newly broken repositories still back live registry skills #193; watch there.
x_ledger_held 134 to 175 (+41): the person-actionable review queue is growing ~5/day while the routine was dark.
Observability gap: 7 check-in days produced no archive (No stale-baseline flag when the routine skips days #195, PR fix(daily-checkin): flag skipped days in the archived window #196 in flight), and the health email probe reads only 2 dates (Health email probe verifies only two Melbourne dates #194, PR fix(daily-checkin): verify a week of health email reports, not two #197 in flight). Both PRs are red on Test.

Proposed actions

Fix the recurring D1 overload 500s on cold-key read paths (issue D1 overload 500s recur after the replayable-read retry fix #192, open from the 2026-09-08 run; no PR yet).
Record source_resolved = 0 when a sync fails on a now-broken repository so dead-backed skills stop presenting live (issue Newly broken repositories still back live registry skills #193, open; no PR yet).
Scope the check-in CI workflow gate to the production ref so PR branch failures stop archiving as a broken main gate (new fingerprint this run).
Land the in-flight observability fixes: No stale-baseline flag when the routine skips days #195 skipped-day flagging (PR fix(daily-checkin): flag skipped days in the archived window #196) and Health email probe verifies only two Melbourne dates #194 seven-date health email probe (PR fix(daily-checkin): verify a week of health email reports, not two #197), both currently red.

Actions 1, 2, and 4 already have open issues or PRs and are not re-proposed; action 3 is new.

The check-in CI collector reads the newest run of any branch, so four PR branch failures archived as a broken main gate while main's Test run on the deployed SHA passed.

Target: scripts/tools/daily-checkin-data.mjs

Verify with: pnpm test:run test/unit/daily-checkin-observability.test.ts

Estimated to change 2 files.

Daily check-in 2026-09-11

AMBER. The front door, deploy, migrations, and job queue are clean, but the D1 overload family keeps firing at scale, three newly broken repositories still back live Skills, and the nightly health email goes AMBER every night on one deleted repository.

Window note: the newest prior archive is 2026-09-02, so this window is about 9.5 days, not one night.

Pulse

Metric	Value	Prior (09-02)
Users	26 (+2)	24
Skills	12,779 (+1,237 new)	11,542
Repos changed in window	485	48
Installs	28	6
Digests sent / failed	0 / 0 (monthly task, next due 2026-10-01)	2 / 0
Known AI cost	$0.00 recorded; unmeasured services unknown	$0.00
X spend	$30.45/mo projected (121/2000 reads today)	$12.30/mo

Embeddings: 1,021 completed, 0 failed, 0 stuck. Skill sync failures: 69. A read-only query splits them 55 path_missing and 14 repo_missing: the registry recording upstream deletions, not sync faults.

X: the projection rose $12.30 to $30.45 over the window, still under the $40 accepted line (issue #160). 6% of the 2,000 daily target used, x_over_budget false, 4 reads per verified skill (escalation line is 6), x_falling_behind false. 121 billed reads against 1,694 posts kept in 24 hours, so no discard gap. The daily budget truncating discovery is the designed steady state, not a fault.

Overnight (deploys since the last run)

No new commits since 2026-09-08. Production runs HEAD 103991d0 as version 266356ed, deployed 2026-09-08T07:40Z. No deploy lag.
Deploy to Cloudflare reads success, read through the skipped guard runs to the run on HEAD.
Test reads failure with 5 consecutive failures. Every failed run sits on a PR branch SHA (fix/issue-198, fix/issue-195, fix/issue-194). The latest Test run on main passed on HEAD 103991d at 2026-09-08T07:30Z. The main gate is green; the collector folds branch runs into the state. Issue CI workflow gate reads PR branch failures as main failures #198 tracks this; PR fix(ops): scope check-in CI reads to main branch runs #199 is open and its own run currently fails build and test.
Three check-in observability fixes sit open with red Test runs: fix(ops): scope check-in CI reads to main branch runs #199 (scope CI reads to main), fix(daily-checkin): verify a week of health email reports, not two #197 (verify a week of health email reports), fix(daily-checkin): flag skipped days in the archived window #196 (flag skipped days). The routine itself skipped 2026-09-03 through 2026-09-10 and nothing flagged it.

Broken

D1 overload family still firing at scale (ledger row 105, grown)

SKILLD-1F, D1_ERROR: D1 DB is overloaded. Requests queued for too long., culprit GET /api/skill-files/davila7/claude-code-templates/data-engineer: 1,030 events, 1 user, first seen 2026-09-02T09:17Z, last seen 2026-09-10T15:36Z. The window holds 21 D1-overload ids, including new SKILLD-24 and SKILLD-23, SKILLD-22, SKILLD-21. Every top culprit names davila7/claude-code-templates, the repository a crawler swept. Hypothesis: cold-key bursts, the documented mechanism, against plain TTL caches or short TTLs. Open PRs #187, #188, #189, #190, #191 cover raw-body, orgs, clusters, tags, and user-skills. The family's largest culprit, skill-files, has no stale shield.

`SKILLD-25` TypeError, new id (new)

SKILLD-25, TypeError: t.toString is not a function, culprit /__nuxt_error.md. 1 event, 0 users, first and last seen 2026-09-11T15:57Z. One event, no user impact, no code evidence yet. Watch one window.

Three newly broken repositories still back live Skills (new)

newly_broken_repos_impacted is 3, from 0. Read-only reads name them: iamzhihuix/happy-claude-skills (13 Skills, broken 2026-09-04T02:02Z), levi-qiao/skillmill (1 Skill, 2026-09-02T15:01Z), aamarva/aamarva-adk (2026-09-02T20:02Z). Verified read-only: GitHub answers 404 for iamzhihuix/happy-claude-skills, so its Skills serve from stored renders behind the source-gone banner. The 80 newly_broken_repos_total and the 69 skill sync failures are the same upstream-deletion wave; most of it is not user-facing.

Nightly health email alarms on a deleted repository every night (new fingerprint)

Both archived operator reports (2026-09-10, 2026-09-11) are AMBER on one reason: "1 reviewed leaderboard repository remained invisible for over 15 minutes: iamzhihuix/happy-claude-skills." That repository is gone upstream, verified 404 today, broken since 2026-09-04. leaderboardStuckSql in layers/identity/server/utils/daily-health-check.ts counts it because it only asks for a healthy repos row, so a reviewed repository deleted upstream alarms forever. The report should carry this once as a deletion, not nightly as a 15-minute stall.

embedding-parity-audit error (new since 09-02)

Last run 2026-09-11T21:02Z records error: "Parity drift: 0 missing, 4 stale, 23 orphan; pruned 23". The audit pruned the 23 orphans; 4 stale vectors remain and failed the run. This is the first non-ok audit since 2026-08-11. The orphan rate fell from roughly 215 per night to 23 in the window, so the leak is draining. Watch one night before acting.

Not broken, stated explicitly

https://skilld.dev/ and /skills both answer 200.
Migration heads match at 0117_email_cadence_split.sql, local and production. No uncommitted migrations, no missing expected tables.
Workers nonOk: 1 scriptThrewException and 13 loadShed against 787,792 successes over the window. Both are low-rate and bounded; the exception sits at the 21:00Z hour where the parity audit throws, matching the ledger's existing attribution. No trend.
All 10 recent job batches finished with failed_jobs: 0. The 84 archived failures split: 81 no_supported_skill_paths (terminal submission decisions, resolved class), 2 repo fetch 404 (last 2026-09-04, matching the deleted repositories above), 1 blob_batch_failed:502 (one-off 2026-09-03). Recovered burst, no active incident.
Health email: both archived report dates exist and were sent. Dates 2026-09-03 to 09-09 sit outside the probe's two-report window (PR fix(daily-checkin): verify a week of health email reports, not two #197).
daily-health-check, send-weekly, recompute-skill-scores, detect-star-surges, and every */5 task ran on cadence with ok. send-digests last ran partial on 2026-09-01, the known paused-AI-summary state from chore(digest): pause the AI summary behind an explicit flag #121; it is next due 2026-10-01.
x_ledger_held moved 134 to 181, oversized holds awaiting review by design.

Drift and observability

No migration, schema, or deploy SHA drift.
Check-in cadence gap: no archives 2026-09-03 to 09-10, and the archived window cannot flag a skipped day (PR fix(daily-checkin): flag skipped days in the archived window #196).
Test workflow state folds PR branch failures into the gate verdict (issue CI workflow gate reads PR branch failures as main failures #198, PR fix(ops): scope check-in CI reads to main branch runs #199 open, itself red).
Health email probe reads two reports, not one week (PR fix(daily-checkin): verify a week of health email reports, not two #197).
AI cost stays unmeasured outside the recorded $0.00 batch estimate.

Proposed actions

Teach leaderboardStuckSql to skip repositories that are broken upstream, so the nightly report stops going AMBER on the same deleted repository. One file plus its spec.
Give the skill-files endpoint the same stale-while-revalidate shield the other swept endpoints are getting, so a crawler sweep over cold keys stops re-running its D1 reads per request. One file plus a test.
Repair and land the three open check-in PRs (fix(ops): scope check-in CI reads to main branch runs #199, fix(daily-checkin): verify a week of health email reports, not two #197, fix(daily-checkin): flag skipped days in the archived window #196); the fixes are written, their Test runs are red.
Watch SKILLD-25 and the parity audit's 4 stale vectors for one window; investigate only on growth.
Keep the X spend row on watch at $30.45/mo against the $40 line; it moved $18 in nine days.

leaderboardStuckSql counts a reviewed repository as invisible whenever no healthy repos row exists, so a reviewed repository deleted upstream (iamzhihuix/happy-claude-skills, 404 verified, broken since 2026-09-04) re-alarms every night and drove both archived reports AMBER.

Target: layers/identity/server/utils/daily-health-check.ts

Verify with: pnpm vitest run layers/identity/server/utils/daily-health-check.spec.ts

Estimated to change 2 files.

The skill-files handler serves from a plain TTL cache with no stale-while-revalidate shield, so crawler sweeps over cold keys re-run the skill lookup, row read, and upstream tree fetch per request and produced 1,030 D1-overload events (SKILLD-1F), the family's largest culprit.

Target: layers/registry/server/api/skill-files/[...slug].get.ts

Verify with: pnpm vitest run test/unit/skill-files-cache.test.ts test/unit/skill-files-tree-outage.test.ts

Estimated to change 2 files.

AMBER: production serves 200s with no active incident, but the Test gate has failed 10 consecutive runs on an offline-runner network fault, leaving two fixes (one aimed at the largest D1 overload culprit) unable to land.

Window note: the previous archive is 2026-09-02, so this window spans ~9.6 days (the routine itself did not run 09-03 to 09-11; that is a scheduler gap, not a production signal). Comparisons below are step changes over that unequal window.

Pulse

Users: 26 (+2). Skills: 12,956 (+1,414). Repos: 8,595 (521 changed). Owners: 9,869.
Installs: 28. Digests: 0 sent / 0 failed (monthly cron; last natural run 09-01).
Known AI cost: $0.00 recorded batch estimate. Unmeasured services are unknown, not $0.
X spend: $30.15/mo projected (122/2000 reads today, 6% used, not over budget, 4 reads per verified skill). Up from $12.30 on 09-02 but under the $40 accepted line (issue X discovery projection passed the accepted spend escalation #160). Discovery is productive: 1,047 verified skills (+551 in window), 1,839 posts in 24h, no discard gap (122 billed reads vs 1,839 survived posts).
Broken inventory: 2,166 cumulative (known). Newly broken in window: 92 total, 5 impacted: ollama/ollama, tinyhumansai/openhuman, iamzhihuix/happy-claude-skills, aamarva/aamarva-adk, levi-qiao/skillmill (all still back live skills).
Health email: sent for both archived Melbourne dates (09-11, 09-12). The 09-12 report is RED: parity-audit failure that night, 21 skills with new sync failures, and the invisible leaderboard repo below.

Overnight (window) changes

Deploy: production on 103991d0 (version d1869c1d, 2026-09-08T07:40Z), equal to origin/main HEAD. No deploy drift. Front door: / and /skills both 200.
CI: Deploy to Cloudflare state success (skipped guard runs read through). Test state failure, 10 consecutive failures. All 10 are PR runs on two branches, latest run 34659286869. Main itself passed: the 09-08 deploy on HEAD succeeded, gated on Test. Logs show one cause: @nuxt/scripts downloads https://static.cloudflareinsights.com/beacon.min.js during vitest transform and the harlan-desktop runner cannot reach it (connect ECONNREFUSED 0.0.0.0:443, timeouts), failing 200 of 251 spec files at import. This blocks the open shield skill-files cache with stale-while-revalidate PR (4 red pushes) and the exclude broken repos from stuck leaderboard alarm PR.
Workers: 862,508 successes; nonOk: 1 scriptThrewException (matches the 09-11 21:00Z parity-audit hour, the pattern from the closed 2026-08-02 row) and 13 loadShed (0.0015%, no attribution in archive, bounded so far).

Broken (new or regressed fingerprints)

Test red on offline runner: beacon.min.js download (new). Evidence: run 34659286869, FetchError [GET] beacon.min.js ... ECONNREFUSED 0.0.0.0:443, 200 files failed / 51 passed, identical across 4 SHAs of the stale-while-revalidate PR and the identity PR. Hypothesis: the runner has no egress to static.cloudflareinsights.com (DNS sinkhole or firewall); the test Nuxt env fetches the beacon because scripts.registry.cloudflareWebAnalytics is registered unconditionally in nuxt.config.ts:200. Fix: stub the script in the test environment so vitest needs no network.
D1 overload family regressed (ledger row of 2026-09-01, watch). Evidence: 22 new issue ids, all D1_ERROR: D1 DB is overloaded, ~1,197 events, firstSeen 09-02T09:17Z (crawler sweep), lastSeen 09-10T15:36Z, after the fix(d1): retry replayable reads on D1 overload #158 replayable-read retry deploy on 09-08. Largest: SKILLD-1F, 1,030 events, culprit GET /api/skill-files/davila7/claude-code-templates/data-engineer. Hypothesis: crawler sweeps traversing the hottest uncached surface (/api/skill-files/...); the open stale-while-revalidate PR is exactly this mitigation and is CI-stuck by item 1. Family stays on watch: exit condition remains a crawler sweep with no burst.
SKILLD-25 TypeError: t.toString is not a function on /__nuxt_error.md (new). Evidence: permalink, 1 event, 0 users, 2026-09-11T15:57Z. Verdict: single event, no reproduction, no user impact; watch, no action yet.
embedding-parity-audit failed one night (new, recovered). Evidence: 09-12 health email, Parity drift: 0 missing, 4 stale, 23 orphan; pruned 23 plus latest_failed history reason, for the 09-11T21:00Z run. The next natural run at 2026-09-12T21:02Z recorded ok, and orphans pruned nightly (23 vs ~215/night in August). Verdict: recovered burst; watch for a second stale-drift night.
Failed jobs: 94 rows in window, of which 91 are terminal no_supported_skill_paths submission decisions (correctly excluded from the health fault count), 2 repo fetch 404 (last 09-04), 1 blob_batch_failed:502 (09-03, single). Latest 10 job batches all finished failed_jobs: 0. Verdict: no active incident.

Drift

Migrations: none. localHead == prodHead == 0117_email_cadence_split.sql, no uncommitted migrations. Missing expected tables: none.
Deploy SHA: none. Production equals origin/main.
Stale tasks: none. daily-health-check last ran 2026-09-11T22:01Z inside its 36h window; all other tasks on cadence; send-weekly ran ok on its Monday schedule (09-07T09:00Z). send-digests still shows the known 09-01 partial (AI credits, paused by chore(digest): pause the AI summary behind an explicit flag #121); next natural run 10-01.
Observability gaps: (a) the workflow-state classifier reads the latest run of each workflow name from any branch, so red PR runs reported the main Test gate as broken for two days while main was green (new finding, action 2); (b) the routine itself produced no archives 09-03 to 09-11, so nine daily health-email dates are unverified by any check-in.

Proposed actions

Stub cloudflareWebAnalytics in the test Nuxt environment (nuxt.config.ts) so vitest never downloads the beacon. This alone turns the Test gate green and unblocks both stuck PRs, including the skill-files stale-while-revalidate shield that targets the 1,030-event D1 overload culprit.
Scope daily-checkin workflow state to main-branch runs in scripts/tools/daily-checkin-data.mjs (with a regression test) so a red PR cannot report the main gate broken.
No action on: parity one-night failure (recovered, watching), SKILLD-25 (1 event, 0 users), loadShed 13 (bounded), X spend ($30.15 under the $40 line), impacted newly-broken repos (upstream churn, correctly surfaced by the health email).

The vitest Nuxt environment downloads beacon.min.js from static.cloudflareinsights.com at transform time, so the network-less harlan-desktop runner fails 200 spec files and the Test gate has been red for 10 consecutive runs.

Target: nuxt.config.ts

Verify with: Push the stub and confirm the next Test run passes: gh run list --workflow Test -R skilld-dev/skilld.dev --limit 1 shows conclusion success

Estimated to change 1 file.

collectWorkflowRuns derives each workflow's state from its latest run on any branch, so ten red PR runs reported the main Test gate broken while main was green and deployed on 2026-09-08.

Target: scripts/tools/daily-checkin-data.mjs

Verify with: pnpm vitest run test/unit/daily-checkin-observability.test.ts with a regression case where a failing PR run follows a passing main run

Estimated to change 2 files.

AMBER: front door is healthy and production matches main with no migration drift, but every open pull request is red on a build-time beacon fetch, the CI archive misreads those red PR runs as the main Test gate, and 7 broken sources still back live skills.

Window: 2026-09-02T01:20Z to 2026-09-13T21:49Z (11 days, the prior archive is 2026-09-02).

Pulse

Users 26 (+2). Skills 13,036 (+1,494 in window). Repos 8,615 (+431). Owners 9,896. Installs 28. Changed repos 542.
Digests 0 sent, 0 failed in window (monthly cadence; last natural run 2026-09-01, AI summary paused by chore(digest): pause the AI summary behind an explicit flag #121).
Known AI cost $0.00 recorded (generation paused; unmeasured services are unknown, not $0).
X spend $22.20/mo projected (86/2000 reads today), under the $40 accepted line, not over budget, 4 reads per verified skill (limit 6). x_posts_24h 1,942 against 86 billed reads: no discard gap.
Parity: embeddings 1,142 completed, 10 failed, 0 stuck. AI submitted/stuck/failed 0 (paused).

Overnight (deploy and CI)

Deploy: version 266356ed live, SHA 103991d0 equals origin/main, deployed 2026-09-08T07:40Z. No deploy drift.
Main Test is green on 103991d0 (run 2026-09-08) and Deploy to Cloudflare succeeded on the same SHA. The archive's Test: failure, 10 consecutive is pull_request runs from fix/issue-204 and fix/issue-205, not main. See Broken.
Deploy to Cloudflare skipped runs on 2026-09-11/12 are the off-main guard working (ledger 2026-08-16 row); they carry no verdict.
Nightly operator email: 2026-09-13 AMBER, sent, reasons: "2 user-impacting repositories became unavailable in 24 hours." and "1 reviewed leaderboard repository remained invisible for over 15 minutes: iamzhihuix/happy-claude-skills." 2026-09-12 RED, sent, reasons: "Scheduled tasks failed: embedding-parity-audit (Parity drift: 0 missing, 4 stale, 23 orphan; pruned 23).", "Scheduled run history is unhealthy: embedding-parity-audit (latest_failed).", "21 skills recorded a new sync failure in 24 hours.", and the same invisible-repo reason. The parity audit is ok again in this archive (last run 2026-09-13T21:01Z).
Workers: 1 scriptThrewException and 13 loadShed against 944,280 successes (0.0015% combined). No path attribution in the dataset; the rate is flat and bounded. No action.
D1 overload family (ledger 2026-08-05 and 2026-09-01 rows) stays chronic: SKILLD-1F holds 1,047 events, lastSeen 2026-09-13T11:51Z, every id crawler-shaped (userCount 1). Still on the existing watch; no new mechanism.

Broken (new or regressed)

Every pull_request build fails fetching the Cloudflare beacon (new). pnpm build dies with FetchError: [GET] "https://static.cloudflareinsights.com/beacon.min.js": <no response> fetch failed (ECONNREFUSED 0.0.0.0:443), 10 consecutive red runs since 2026-09-11 (example run 34721728017 on fix/issue-204). Hypothesis: the harlan-desktop runner's resolver blackholes static.cloudflareinsights.com, and @nuxt/scripts fetches the beacon at build time with no offline fallback; main's 2026-09-08 build predated the block. Blocked merges have frozen main since 2026-09-08, so nothing has shipped for 5 days. Issue Test suite downloads Cloudflare beacon on offline runner #204 is open; its in-flight fix is itself red on the same build step.
CI gate reads red pull_request runs as the main Test gate (new). summarizeWorkflowRuns (scripts/tools/daily-checkin-observability.mjs) filters by workflow name only, so 10 red PR runs read as Test: failure, consecutiveFailures: 10 while the latest main run passed (verified read-only: gh run list --workflow Test --branch main). Same class as the 2026-08-16 skipped-deploy row. Issue CI gate state reads red PR runs as main Test failures #205 is open with an in-flight fix.
SKILLD-25 TypeError: t.toString is not a function (new, non-D1). Permalink, culprit /__nuxt_error.md, 1 event, 0 users, first and last seen 2026-09-11T15:57Z. Hypothesis: the error page failed while rendering an error for an unusual .md URL. Watch only; no action on a single event.
7 broken sources still back live skills (regressed from 0). newly_broken_repos_impacted 0 → 7 over the window (102 newly broken total): skillsgate/skillsgate, thinhvu/frontier-engineering, ollama/ollama, tinyhumansai/openhuman, iamzhihuix/happy-claude-skills, aamarva/aamarva-adk, levi-qiao/skillmill, each still backing at least one live skill. Both nightly reports name iamzhihuix/happy-claude-skills (reviewed leaderboard repo) invisible for over 15 minutes. 89 skill sync failures and 10 embedding failures in the window correlate with the same breakage wave.

Drift

Migrations: local head 0117_email_cadence_split.sql = prod head. No uncommitted migrations. No missing expected tables.
Deploy SHA equals origin/main. The freeze is upstream of production: main cannot move until the PR build defect is fixed.
No stale reserved jobs, no stale scheduled tasks, all sync jobs ok except send-digests, whose partial row is stale state from 2026-09-01 (pre-pause; monthly cron has not run since chore(digest): pause the AI summary behind an explicit flag #121 deployed). Next natural run 2026-10-01 confirms the pause.
Observability: no new gaps. Sentry probe not truncated. No failed digest, no failed health-email delivery (one sent per Melbourne date, 2026-09-12 and 2026-09-13).

Proposed actions

Fix the build-time beacon fetch so pnpm build passes with static.cloudflareinsights.com unreachable, land it, and confirm a merge turns main Test + Deploy green (issue Test suite downloads Cloudflare beacon on offline runner #204).
Make the CI gate read main-branch runs only, so red pull_request runs stop reporting as the repository Test gate (issue CI gate state reads red PR runs as main Test failures #205); repair the in-flight PR, whose build currently fails with action 1.
Read-only triage of the 7 impacted broken sources: for each, decide wait-for-recovery vs removal to the 410 tombstone. Prioritise iamzhihuix/happy-claude-skills (leaderboard-reviewed, invisible two nights running) and ollama/ollama. No production mutation from the check-in.

The Nuxt build fetches https://static.cloudflareinsights.com/beacon.min.js at build time with no offline fallback, so every pull_request build fails when the harlan-desktop runner blocks that host, freezing all merges since 2026-09-11.

Target: nuxt.config.ts

Verify with: pnpm build with static.cloudflareinsights.com resolving to 0.0.0.0, then confirm main Test and Deploy to Cloudflare pass on the merged SHA

Estimated to change 3 files.

summarizeWorkflowRuns filters runs by workflow name only, so red pull_request runs report as the repository Test gate (failure, 10 consecutive) while the latest main-branch run passed.

Target: scripts/tools/daily-checkin-observability.mjs

Verify with: pnpm vitest run test/unit/daily-checkin-observability.test.ts

Estimated to change 2 files.

AMBER. Production is current with main, CI and both deploys are green, migrations match, and the D1 overload family has not fired since 2026-09-13 11:51Z, but the 12.9-day window accumulated 7 newly broken repositories that still back live skills, pending fix under issue #193, and the X spend projection sits $0.40 under the accepted $40 escalation line.

Baseline note: the routine skipped 12.9 days (skilld.baseline reports stale, gapHours 308.4, the flag #196 added). Every rate below is per day, never an overnight step change.

Pulse

Metric	Value	Prior (2026-09-02 window)
Users	26 (+2 new)	24
Skills	13,127 (+1,585 new, ≈123/day)	11,542 (+62/day)
Repos changed in window	573 (≈45/day)	48/day
Installs	28 (≈2.2/day)	6/day
Digests sent / failed	0 / 0 (monthly ran 09-01, outside window; `send-weekly` ok 09-14)	2 / 0
Known AI cost	$0.00 recorded (`ai_submitted` 0, summary paused by #121)	$0.00
X spend	$39.60/mo projected (165/2000 reads today, 8% used)	$12.30/mo

Embeddings: 1,243 completed, 11 failed (0.9%), 0 stuck. skill_sync_failures 94 and 124 newly broken repos total feed the 7 impacted ones, identities archived by the #166 probe: skillsgate/skillsgate, thinhvu/frontier-engineering, ollama/ollama, tinyhumansai/openhuman, iamzhihuix/happy-claude-skills, aamarva/aamarva-adk, levi-qiao/skillmill.

X spend detail: x_projected_monthly_usd is 39.60 against the $40 accepted line from issue #160, so no escalation, but the headroom is $0.40. x_over_budget false, reads per verified skill 4 (limit 6). Billed reads today 165 sit below the 2,116 posts kept in 24 hours, so the query is not paying for discarded posts. Existing issue #179 carries the tuning decision.

Overnight (2026-09-02 → 2026-09-14)

55 commits landed since the last run. The 2026-09-14 wave deployed the D1 and cache repairs (fix(d1): retry replayable reads on D1 overload #158, fix(cache): serve stale cached responses when a recompute fails #168, fix(registry): serve last-good raw body during upstream outages #187-fix(api): cache user skills route reads per login #191), the check-in collector fixes (fix(daily-checkin): flag skipped days in the archived window #196, fix(daily-checkin): verify a week of health email reports, not two #197, fix(ops): scope check-in CI reads to main branch runs #199, fix(identity): exclude broken repos from stuck leaderboard alarm #202), the beacon fixes (fix(scripts): skip cloudflare beacon bundling under vitest #207, fix(scripts): fall back to remote beacon when build-time fetch fails #211), and refactor(checkin): use registered checks and shared CLI #213/perf(registry): refresh skill-live audits once a day #216/fix(sentry): drop benign closed-channel postMessage error on client #217.
Production: version deployed 2026-09-14T17:33:09Z at approxDeployedSha b828b67, equal to origin/main HEAD. No deploy lag. The freeze from Production frozen 12 commits and 4 days behind main #144 is over.
CI: Test state success on b828b67 (run 34873876957). Deploy to Cloudflare state success read through to run 34875062789; the many skipped guard runs in front of it are read through correctly.

Broken

D1 overload family grew inside the window, quiet after the current deploy (known, #192/#201)

21 of the 22 new-window Sentry issues are the D1_ERROR: D1 DB is overloaded family, led by SKILLD-1F at 1,047 events on GET /api/skill-related/openai/openai-agents-python/runtime-behavior-probe, first seen 2026-09-02T09:17:07Z, last seen 2026-09-13T11:51:50Z. SKILLD-1Y/1W/1X and friends are the same error on skill detail, skill-files, orgs, tags, collections, and stats culprits.

The family's lastSeen (2026-09-13T11:51Z) predates the current deploy (2026-09-14T17:34Z), and this archive's window ends 2026-09-14T21:42Z with no post-deploy event. That is not a regression of the #158/#168 fixes; it is the evidence the open issues need. #192 tracks the recurrence class, #201 names skill-files as the largest culprit, and PR #203 implements its stale-while-revalidate shield.

Hypothesis, unchanged: cold-key crawler sweeps over uncached endpoints re-run D1 reads per request. PR #203 is the pending fix for the biggest path.

`SKILLD-25` TypeError on the built-in error page (new id, watch)

SKILLD-25, TypeError: t.toString is not a function, culprit /__nuxt_error.md. 1 event, 0 users, first and last seen 2026-09-11T15:57:00Z, on the pre-deploy version. Event detail read read-only and appended to the archive: the crash is Vue's display-string helper inside nuxt-error-page.vue setup, after a navigation to https://skilld.dev/__nuxt_error.md with console error [NUXT_E1005].

Hypothesis: the error payload handed to the built-in error page carried a non-string status/message, so rendering called toString on it. One event, zero users, no recurrence in three days, framework-internal. Watching before proposing code.

Not broken, stated explicitly

https://skilld.dev/ and /skills both 200. The protected health report gate passed with nothing missing, failed, warned, or unavailable.
Migration heads match at 0117_email_cadence_split.sql, local and production. No uncommitted migrations, no missing expected tables.
workers.nonOk: 1 scriptThrewException and 13 loadShed over 12.9 days, about one shed request per day against 1,051,174 successes. Ledger rows 34 and 53 both closed the same shape as bounded bursts. 844 clientDisconnected is navigation noise. Monitor, no action.
126 failed_jobs rows: 122 are no_supported_skill_paths submission decisions (ledger row 58 class, excluded from fault counts), 3 repo fetch 404 are terminal deleted-repo verdicts (ledger row 89 class), 1 blob_batch_failed:502 dates to 2026-08-11. All ten recent job batches finished with failed_jobs: 0, latest registry-sync:1789419645. Recovered, not active.
send-digests partial is the known paused-summary state (chore(digest): pause the AI summary behind an explicit flag #121), bounded by fix(health): bound partial sync job alarm to the 24h report window #165, monthly cron, next natural run 2026-10-01. No stale scheduled task, no stale reserved jobs, no failed digest.
Cumulative broken_repos 2,195 is known inventory, not a window event.
The CI collector scoped to main (fix(ops): scope check-in CI reads to main branch runs #199) and the stale-baseline flag (fix(daily-checkin): flag skipped days in the archived window #196) both worked as designed in this run.

Drift and observability

Baseline stale by 308.4 hours. Rates above are per day. The next runs should return to the daily cadence or pass an explicit --since.
No migration, schema, or deploy SHA drift.
Observability gaps: none new. Deploy workflow sampling, main-branch CI scoping, and impacted-repo identities all behaved.

Proposed actions

Land PR fix(registry): shield skill-files cache with stale-while-revalidate #203 (fix(registry): shield skill-files cache with stale-while-revalidate) to close issue skill-files cache lacks a stale shield against cold bursts #201, the family's largest culprit. Review and merge, then verify with pnpm vitest run test/unit/skill-files-cache.test.ts test/unit/skill-files-tree-outage.test.ts. Next actor: Harlan's review on PR fix(registry): shield skill-files cache with stale-while-revalidate #203.
Implement issue Newly broken repositories still back live registry skills #193 (Newly broken repositories still back live registry skills): the window's 7 impacted identities are archived evidence. Verify with pnpm test:run test/unit/sync-repo-correctness.test.ts. Next actor: triage-to-implementation on issue Newly broken repositories still back live registry skills #193.
Decide the X discovery budget on issue X discovery projects past the accepted $40 monthly spend line #179: the projection moved $42.60 → $39.60, $0.40 under the accepted $40 line, at 264 projected reads/day against the issue's proposed ~250 cap. Raise the line deliberately or lower DAILY_DISCOVERY_READ_BUDGET in shared/server/x-ingest.ts. Next actor: Harlan, on issue X discovery projects past the accepted $40 monthly spend line #179.
Watch D1 overload for one post-deploy crawler sweep (issue D1 overload 500s recur after the replayable-read retry fix #192, ledger row): the exit condition is a traffic peak like 2026-09-05T07Z passing quietly on the current version. Next actor: the next daily check-in.
Watch SKILLD-25: if a second event lands post-deploy, fix the error page payload typing; until then it is one framework-internal event from a retired version. Next actor: the next daily check-in.

Agent triage by Codex, 2026-10-05. This is an agent assessment.

This retired run log is superseded by individual daily reports. Outstanding follow-up remains in #179, #292, and #393. Closing this log does not disable the routine.

Harlan Agent Kit updates the latest report in this issue body. Each run remains in the private archive and Dashboard history.

[ACTION NEEDED] Daily check-in: skilld-dev/skilld.dev #126

Description

[ACTION NEEDED] Daily check-in: 2026-10-11

AMBER skilld.dev

Activity

harlan-github-agent commented on Sep 3, 2026

harlan-github-agent commented on Sep 3, 2026

Daily check-in 2026-09-03

Pulse

Overnight

Broken

pnpm build killed with exit 129 on the self-hosted runner (new)

D1 overload crawler sweep, recurrence (watch row 2026-09-01)

Two user-impacting repositories newly broken (0 → 2)

Not broken, stated explicitly

Drift and observability

Proposed actions

harlan-github-agent commented on Sep 3, 2026

Daily check-in 2026-09-03

Pulse

Overnight

Broken

Deploy freeze since 2026-09-02T01:31Z (new)

ci probe 401 Bad credentials (new)

SKILLD-1E grew on a new culprit while its fix sits undeployed (regressed)

2 user-impacting repos newly broken (new, verified benign)

D1 overload burst recurred on new ids (known family, ledger rows 45 and 105)

Not broken, stated explicitly

Drift and observability

Proposed actions

harlan-github-agent commented on Sep 4, 2026

Daily check-in 2026-09-04

Pulse

Overnight

Broken

Deploy frozen: the CLI grammar gate cannot read a one-day-old release (new)

D1 overload family recurred across the whole window (regressed, known family)

SKILLD-1E grew while its fix sits in the frozen backlog (regressed)

SKILLD-1V KV PUT 429 escaped after the writeCache class fix (new id, known class)

Not broken, stated explicitly

Drift and observability

Proposed actions

harlan-github-agent commented on Sep 6, 2026

Pulse

Overnight

Broken

Drift

Proposed actions

harlan-github-agent commented on Sep 6, 2026

Pulse

Overnight

Broken

Verdicts on recovered or bounded signals

Drift

Proposed actions

harlan-github-agent commented on Sep 7, 2026

RED: Test is red on main and deploys are frozen behind HEAD, and X discovery now projects $42.60/mo past the $40 accepted line.

Pulse

Overnight: deploy and CI

Broken

Drift

Proposed actions

harlan-github-agent commented on Sep 8, 2026

RED: X discovery projects $76.20/mo past the $60 red line, the D1 overload family threw again after the #158/#168 fixes deployed, and 3 newly broken repositories still back live skills.

Pulse

Overnight

Broken

Drift

Proposed actions

harlan-github-agent commented on Sep 9, 2026

Daily check-in 2026-09-09

Pulse

Overnight

Broken

D1 overload family recurred across the dark week (regressed, ledger rows 45 and 105)

X discovery projection crossed the accepted $40 line (gate breach, ledger row 59)

Three newly broken sources still back skills (impacting gate, new identities)

The check-in itself was dark for six days (observability, new)

Not broken, stated explicitly

Drift and observability

`pnpm build` killed with exit 129 on the self-hosted runner (new)

`ci probe 401 Bad credentials` (new)

`SKILLD-1E` grew while its fix sits in the frozen backlog (regressed)

`SKILLD-1V` KV PUT 429 escaped after the `writeCache` class fix (new id, known class)

RED: `Test` is red on `main` and deploys are frozen behind HEAD, and X discovery now projects $42.60/mo past the $40 accepted line.

`SKILLD-25` TypeError, new id (new)

`SKILLD-25` TypeError on the built-in error page (new id, watch)