Skip to content

fix(ops): sample deeper when skipped guard runs hide the deploy verdict - #118

Merged
harlan-zw merged 2 commits into
mainfrom
fix/checkin-deploy-run-sample
Sep 4, 2026
Merged

harlan-zw merged 2 commits into
mainfrom
fix/checkin-deploy-run-sample

Conversation

@harlan-zw

Copy link
Copy Markdown
Collaborator

❓ Type of change

  • 🐞 Bug fix

📚 Description

The 2026-09-02 check-in archived Deploy to Cloudflare as missing while the deploy for that SHA had already succeeded and production was serving it. missing is meant to say the check-in could not see a verdict, so firing it on a healthy deploy teaches the reader to skip the line.

The sample was the problem, not the classifier. A workflow_run event from every branch triggers the deploy workflow and its guard skips itself off main, so the 10 runs the collector asked for on 2026-09-01 were all guard runs:

0   33530550274  skipped  2026-09-01T16:13:49Z
..
10  33476392865  skipped  2026-09-01T06:09:18Z
11  33476257524  success  2026-09-01T06:07:18Z   <- the deploy, one row past the page

It now asks for 100 runs, but only for a workflow whose first page came back full and carried no verdict, so the ordinary case still costs one request each.

I wanted to ask GitHub for the last non-skipped run rather than pick a number. It cannot answer that: gh run list --status and the REST ?status= filter both take a single value, and a verdict can be any of seven conclusions. The exclusion stays client side and both numbers are commented where they live.

Open question I did not resolve: 100 is one GitHub page and it keeps missing honest, but I have no measurement of how long a skipped streak gets on a busy day. If one ever passes 100, this comes back as missing just as quietly.

Ledger row added under Deploy verdict falls off the end of the run sample.

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

https://claude.ai/code/session_013pSLxovqDjiP7U7rUrRh9n

The 2026-09-02 check-in archive reported `Deploy to Cloudflare` as `missing`
while the deploy for that SHA had succeeded. The classifier was right and the
sample was too shallow.

`daily-checkin-data.mjs` asked `gh run list --workflow <name> --limit 10`. A
`workflow_run` event from every branch triggers the deploy workflow, so its
guard writes a `skipped` run on every push. On 2026-09-01 eleven consecutive
skipped runs sat in front of the success, one row past the page edge, so the
sample carried no verdict at all.

GitHub cannot answer "the last run that was not skipped". Both `gh run list
--status` and the REST `?status=` filter take one value, while a verdict can be
success, failure, cancelled, timed_out, neutral, action_required or stale. The
exclusion has to be a client-side read of ordered history.

`collectWorkflowRuns` keeps the 10-run head page and asks once more for 100 runs
only when that page was full and reached no verdict. A short head page is the
workflow's whole history, so it is never re-fetched. 100 is one GitHub page, so
a workflow with no verdict in 100 runs has genuinely not reported one and
`missing` stays meaningful.

Claude-Session: https://claude.ai/code/session_013pSLxovqDjiP7U7rUrRh9n
@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. labels Sep 2, 2026
@harlan-github-agent

harlan-github-agent Bot commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

🤖 MERGED

Harlan Agent Kit posted this automated review. It is not Harlan's personal review or approval. AI open source policy. Last updated: 2026-09-04 05:13 UTC.

GitHub merged this pull request. No further automated Review will run.

@harlan-github-agent harlan-github-agent Bot added harlan-agent-ready The automated Review passed every gate on this head commit. harlan-agent-blocked The automated Review found a material defect in this head commit. and removed harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-review-required Pull request triage requires an adversarial Review for this head commit. harlan-agent-ready The automated Review passed every gate on this head commit. labels Sep 2, 2026
@harlan-github-agent harlan-github-agent Bot added harlan-agent-running An Agent holds a Task on this issue or pull request right now. harlan-agent-pending The automated Review is waiting on a gate for this head commit. and removed harlan-agent-blocked The automated Review found a material defect in this head commit. harlan-agent-running An Agent holds a Task on this issue or pull request right now. labels Sep 4, 2026
@harlan-zw
harlan-zw merged commit 5d2e651 into main Sep 4, 2026
8 of 10 checks passed
@harlan-github-agent harlan-github-agent Bot removed the harlan-agent-pending The automated Review is waiting on a gate for this head commit. label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant