Skip to content

fix(list-recent-runs): anchor completion window to intended cron tick - #526

Merged
max-sixty merged 2 commits into
mainfrom
hourly/review-25954184316
May 18, 2026
Merged

max-sixty merged 2 commits into
mainfrom
hourly/review-25954184316

Conversation

@tend-agent

@tend-agent tend-agent commented May 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Anchor list-recent-runs.sh's completion window to the most recent intended cron tick (read from $GITHUB_EVENT_PATH) instead of now. Consecutive review-reviewers cycles then tile exactly: [intended-1h, intended] → [intended, intended+1h], regardless of GHA scheduler delay.

Replaces an earlier 2h-widening approach per maintainer feedback (#issuecomment-4472767685). Strictly cleaner: no overlap between consecutive cycles, no slack between them.

Background

The hourly review-reviewers schedule (cron 47 * * * *) is delayed 20–40 min during peak hours, producing back-to-back cycle gaps that systematically exceed the script's 1h now-anchored window. Last 11 review-reviewers runs on max-sixty/tend (oldest → newest), interval to next:

Started Gap to next
2026-05-15T17:16Z 59 min
2026-05-15T18:15Z 60 min
2026-05-15T19:15Z 58 min
2026-05-15T20:13Z 59 min
2026-05-15T21:12Z 57 min
2026-05-15T22:09Z 60 min
2026-05-15T23:09Z 56 min
2026-05-16T00:05Z 77 min
2026-05-16T01:22Z 81 min
2026-05-16T02:43Z 82 min
2026-05-16T04:05Z 99 min

The four most recent intervals exceed 1h. In the analysis run 25954184316 (gap 99 min from the prior review-reviewers), the script returned 4 of 17 in-window tend runs, missing PR #523 (release 0.0.21), PR #524 (regen workflows), and the 25952710024 tend-review-runs failure on cutover-to-codex — substantive activity recovered only via a direct gh run list sweep.

How it works

schedule=$(jq -r '.schedule // empty' "$GITHUB_EVENT_PATH")  # "47 * * * *"
# Match simple hourly crons (MM * * * *); extract minute.
# Compute the most recent matching tick ≤ now; that's the intended fire time.
COMPLETED_AFTER=$((intended - 3600))
CREATED_SINCE=$((intended - 10800))  # keep the 2h buffer for long runs

Non-schedule events (workflow_dispatch, ad-hoc CLI use) and non-hourly crons fall back to the original now - 1h window — same behavior as before the change.

Trade-offs

  • No overlap between consecutive cycles (vs. the 2h approach, which would re-analyze each run on overlapping cycles — cheap but messy).
  • Skipped-tick edge case: if a cron tick fires zero times (rare under sustained outage), that hour's completions fall in the gap between the previous and next cycle's windows. The 2h approach would have caught them; this one doesn't. Considered acceptable per the comment thread.
  • Daily/multi-value crons (0 4 * * *, 0,30 * * * *) keep the now-anchored fallback. Could extend later if needed.

🤖 Generated with Claude Code

…lack

review-reviewers' hourly schedule (cron `47 * * * *`) is delayed 20–40 min
during peak hours, producing 80–100 min gaps between consecutive cycles.
The 1h `updatedAt` cutoff in list-recent-runs.sh dropped everything older
than the previous cycle's expected start, silently hiding tend runs that
landed in the delay slack.

Bump the cutoff to 2h. Covers the largest observed gap (99 min) with
margin; the small re-analysis overlap between cycles is cheap because
gist entries are keyed by run ID.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@tend-agent tend-agent added the claude-behavior Behavioral issues found by review-reviewers label May 16, 2026
@max-sixty

Copy link
Copy Markdown
Owner

do we get an input of the intended cron time, which we can use to look over exactly one hour of time?

@tend-agent

Copy link
Copy Markdown
Collaborator Author

Not directly — GHA doesn't expose an intended-fire timestamp on the schedule event. The only related field is github.event.schedule, which carries the cron expression (e.g. "47 * * * *"), not when it was supposed to fire (GHA events docs).

But the intended time is derivable from that expression + now: it's the most recent matching tick ≤ now. For our 47 * * * * cron that's just the most recent :47:00 UTC. Anchoring the window to intended (rather than actual start) lets consecutive cycles tile exactly: [intended-1h, intended], then [intended, intended+1h]. No overlap, no slack — strictly cleaner than the 2h widening this PR ships.

Sketch:

THIS_MIN_TS=$(date -u -d "$(date -u +%Y-%m-%dT%H:00:00) 47 minutes" +%s)
if [ "$(date -u +%s)" -lt "$THIS_MIN_TS" ]; then
  INTENDED=$((THIS_MIN_TS - 3600))
else
  INTENDED=$THIS_MIN_TS
fi
CREATED_SINCE=$(date -u -d "@$((INTENDED - 3600))" +%Y-%m-%dT%H:%M:%S)
COMPLETED_AFTER=$((INTENDED - 3600))

Caveats:

  • workflow_dispatch and other non-schedule triggers have no event.schedule — would need a fallback to now - 1h.
  • Parsing arbitrary cron is messy; hardcoding 47 (or passing it via env from the workflow) is the pragmatic move since the script is review-reviewers-specific.
  • Edge case: if a cron tick is skipped entirely (rare but possible under sustained delay), one cycle's worth of completions falls into the gap. The 2h window in this PR would catch them; the intended-time window wouldn't.

Want me to switch the PR to the intended-time approach? Happy to either replace the 2h bump or layer the intended-time anchor on top of a smaller (e.g. 75 min) safety margin.

@max-sixty

Copy link
Copy Markdown
Owner

yes please, and remove the 2H window change

Replaces the 2h widening with an intended-tick anchor: when the script
runs under a scheduled workflow with a simple hourly cron (`MM * * * *`),
read the cron from $GITHUB_EVENT_PATH and anchor the window to the most
recent matching tick instead of `now`. Consecutive cycles tile exactly
([intended-1h, intended]) regardless of GHA scheduler delay, with no
overlap. Non-schedule events and non-hourly crons fall back to the
original now-anchored 1h window.

Per maintainer feedback on PR #526 — strictly cleaner than the 2h
widening it replaces.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@tend-agent tend-agent changed the title fix(list-recent-runs): widen completion cutoff to 2h for cron-delay slack fix(list-recent-runs): anchor completion window to intended cron tick May 18, 2026
@tend-agent

Copy link
Copy Markdown
Collaborator Author

Done — pushed fdc947a. Auto-detects the cron from $GITHUB_EVENT_PATH so no skill/workflow change needed; matches simple hourly crons (MM * * * *) and falls back to the existing now - 1h window for everything else (workflow_dispatch, daily crons, etc.). PR title and description updated.

@max-sixty
max-sixty merged commit b4e06e4 into main May 18, 2026
4 checks passed
@max-sixty
max-sixty deleted the hourly/review-25954184316 branch May 18, 2026 14:21
@max-sixty max-sixty mentioned this pull request May 19, 2026
3 tasks
max-sixty added a commit that referenced this pull request May 19, 2026
## Summary

Release 0.0.25. 8 PRs since 0.0.24:

- #566 review: push fixes to bot PRs by default, even when the fix needs
a judgment call
- #563 skills(running-tend): drop stale session-log count
- #562 skills: drop dollar-amount cost asides corrupted by arg
substitution
- #526 fix(list-recent-runs): anchor completion window to the intended
cron tick
- #559 fix(skills): hard-code /tmp instead of $TMPDIR in worktree
recipes
- #560 fix(enrich-tend-outage-issues): batch into one comment per issue
- #557 config: switch review-reviewers back to claude, demote
codex-auth-refresh to reference
- #556 config: switch tend's own harness back to claude

#566 is the reason for cutting now: it flips the bot-PR review default
from "push only mechanical fixes" to "push the fix whenever the review
can articulate it, defer only when no defensible default exists." The
behavior is release-gated, so it only reaches tend's own dogfooded
workflows (and adopters like PRQL) once 0.0.25 is published and the
nightly regen repins the action ref.

No new breaking changes since 0.0.24.

## Test plan

- [ ] CI green (test, test-worker, lint)
- [ ] After merge: tag `0.0.25`, PyPI publish (`pypi-release.yaml` on
the tag push), `uvx tend@0.0.25 --help` works
- [ ] Regenerate tend's own workflows to `@0.0.25` (separate `chore:
regenerate` PR, after the tag exists)

Co-authored-by: Claude <noreply@anthropic.com>
max-sixty pushed a commit that referenced this pull request Jul 10, 2026
…chor (#753)

## Problem

`list-recent-runs.sh` anchors its completion window to the most recent
intended cron tick and floors it at `intended - 3600`. This tiles
exactly — `[intended-1h, intended]` then `[intended, intended+1h]` —
**only when consecutive cron ticks actually fire.** GHA doesn't just
*delay* scheduled ticks (which the tick-anchoring in #526 handles well);
it *drops* them. A dropped tick leaves that hour's completions in the
gap between the previous and next cycle's windows, and no run ever
analyzes them.

[#526](#526) named this exact
"skipped-tick edge case" and deferred it as "rare under sustained
outage," because the only fix on the table then was the 2h-overlap
approach the maintainer had rejected for re-analyzing runs. This run is
direct evidence it is **not** rare and the miss is **invisible** — a
dropped tick produces no error, so the skipped outcomes just never get
audited.

## Evidence (this run)

Analyzing `max-sixty/worktrunk`, `review-reviewers` fired at `07:50Z`
(07:47 tick) then `09:46Z` (09:47 tick) — **GHA dropped the 08:47
tick.** The window for the 09:47 cycle floored at `08:47`, leaving
`[07:47, 08:47]` covered by no cycle. `list-recent-runs.sh` returned
**3** of the **10** tend-* runs actually in `[07:47, 09:47]`. The
orphaned hour contained real, substantive bot output:

-
**[worktrunk#3355](max-sixty/worktrunk#3355
— a well-sourced answer on Windows Defender code-signing (comment
`4874219842`, `08:16Z`).
- **[worktrunk#3357](max-sixty/worktrunk#3357
— a full review cycle: `COMMENTED` flagging `codecov/patch`, author
pushed coverage, bot `APPROVED` on the fixed commit (`08:15Z`–`08:23Z`).

Both were healthy outcomes this time — but had either gone wrong, no
`review-reviewers` cycle would have caught it. (I covered the gap
manually for this run's audit; the gist records the full window.)

## Fix

Floor the window at the previous **actual** completed run's intended
tick instead of assuming the previous tick fired. #526's discussion
weighed only two options — the *intended* fire time (not exposed by GHA)
vs 2h overlap (rejected) — and missed a third: the previous run's own
start time **is** queryable, and rounding it down to its cron tick
recovers exactly where the last cycle's window ended.

- **Every tick fires** → previous intended tick `== intended - 3600` →
byte-identical to today's behavior, **no overlap** (the constraint #526
set).
- **A tick was dropped** → floor reaches back to the last actual run's
tick, covering the orphaned hour with no double-analysis.
- Capped at **6h** reach-back so a sustained outage can't create an
unbounded window. Falls back cleanly (`gh` failure, no
`GITHUB_WORKFLOW`) to the current floor.

Verified in this CI session: the patched script returns all **10**
in-window runs (was 3); with consecutive ticks it resolves the floor to
`intended - 3600` unchanged.

## Gate assessment

- **Type:** targeted fix to one branch of the window logic (not a
structural rewrite).
- **Classification:** structural + deterministic — a dropped tick
*always* produces the gap; not a stochastic model lapse.
- **Occurrences:** 1 confirmed gap-with-missed-outcomes this run. Below
the normal 2–3 bar, but acted on because the failure mode is
**self-concealing**: future cycles use the same window and won't
re-detect the gap, so evidence cannot accumulate — "wait for occurrence
2" would sit at 1 while silently under-covering. This is tend's own
audit tooling, the fix reduces to a no-op in the common case, and human
review is the backstop.

Evidence log: https://gist.github.com/b923bbc52f3c6606931d59abdae616a3

Co-authored-by: tend-agent <tend-agent@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
max-sixty pushed a commit that referenced this pull request Aug 12, 2026
… and stop the fetch limit re-truncating it (#838)

`list-recent-runs.sh`'s dropped-tick recovery anchors the window floor
on the previous **completed** run of the analyzing workflow. `completed`
is a status, not a conclusion, so a run that failed before its agent
produced any analysis advances the anchor exactly as if it had covered
its hour. After an outage the next green run therefore resumes one tick
back and reports an all-clear for a window nothing ever looked at — and
the gap is unrecoverable, because the following run's floor advances
past it too.

This is not hypothetical: it just consumed ~15 hours of
`review-reviewers` coverage.

## What happened

Every `review-reviewers` run from
[30897445507](https://github.com/max-sixty/tend/actions/runs/30897445507)
(09:41Z) through
[30959115609](https://github.com/max-sixty/tend/actions/runs/30959115609)
(23:10Z) failed — 16 consecutive runs, all five matrix legs, `claude -p`
exiting 1 after ~4s. The `result` event in every leg's session log is
identical: `is_error: true`, `num_turns: 1`, `result: "You've hit your
weekly limit · resets 12am (UTC)"`, with `token-usage.json` all zeros.
The agent never started, so none of those runs analyzed anything. (The
outage itself is already tracked by #831 and the annotation-legibility
work in #816 / #818 — this PR is only about the window the outage
swallowed.)

The last run that did analyze anything is
[30893072876](https://github.com/max-sixty/tend/actions/runs/30893072876)
at 08:42Z, which is also the last entry in the `numbagg/numbagg`
evidence gist. At this run's tick (23:47Z intended), the recovery query
picked
`{"conclusion":"failure","createdAt":"2026-08-04T23:10:57Z","databaseId":30959115609}`
and floored the window at 22:47Z, so `list-recent-runs.sh` returned
exactly one row: a `tend-notifications` pre-check no-op.

Everything `numbagg-bot` actually did in the swallowed window sat
outside that floor — PRs
[#719](numbagg/numbagg#719) and
[#720](numbagg/numbagg#720), issue
[#721](numbagg/numbagg#721), three reviews and
four inline comments on #719, and comments on #716, #721 and #1. That is
the densest window numbagg has had in days, and per the skill's Step 1
("If empty, record the run as all-clear … then skip to Step 6") it would
have been recorded as a quiet hour. I only found it by diffing the run
list against the gist's last recorded boundary by hand.

## The change

Anchor on the previous **successful** run rather than any completed one,
filtering server-side with `--status success` (the flag takes
conclusions as well as statuses), and when the existing 6h cap clamps
the recovered floor, say so on stderr so the caller records a coverage
gap instead of a false all-clear. A partially-failed matrix run counts
as a failure here, which only ever widens the window — overlap is
re-offered work the caller dedups against its own evidence log, whereas
a gap is silently unanalyzed.

Widening the window then exposed a second truncation one layer down, so
the fetch loop moves with it. It passed `--limit 50` while a recovered
window spans up to 8h, and `gh run list` returns newest-first — so a
workflow over the limit silently drops its *oldest* runs, which are
exactly the ones in the gap the anchor just reached back for. On
`max-sixty/tend`, `tend-mention` alone produces 57 runs in an 8h window.
The limit is now 200, and a workflow returning exactly the limit warns
rather than truncating in silence.

Two smaller pieces: the anchor query routes through `gh_retry` and exits
non-zero instead of `2>/dev/null || true`, so a transient API error can
no longer masquerade as "no successful run" and emit a confident warning
naming a cause that didn't happen; and `review-reviewers/SKILL.md`,
which still said "if empty, record all-clear" with nothing about the
stderr warnings, now treats any `WARNING:` as a coverage gap — without
that, an agent following the skill literally would print the warning and
record an all-clear anyway.

When every tick fires and succeeds the anchor is the previous tick,
`prev_intended == intended - 3600 == COMPLETED_AFTER`, and the
comparison is a no-op — output is byte-identical to today's on the
healthy path.

<details><summary>Verification against live data</summary>

Same tick, same repo, before vs. after:

```
# anchor query — before (--limit 10, no conclusion filter)
{"conclusion":"failure","createdAt":"2026-08-04T23:10:57Z","databaseId":30959115609}

# anchor query — after (--limit 50, conclusion == "success")
{"conclusion":"success","createdAt":"2026-08-04T08:42:12Z","databaseId":30893072876}
```

`TARGET_REPO=numbagg/numbagg ./list-recent-runs.sh` after the change
returns 5 runs instead of 1, preceded by:

```
WARNING: the last successful 'review-reviewers' run started 2026-08-04T08:42:12Z, more than 6h back. Window floored at 2026-08-04T17:47:00Z; runs that completed before it are NOT in this list. Record a coverage gap, not an all-clear.
```

The recovered anchor (08:42:12Z) matches the gist's last recorded
coverage boundary exactly. Also exercised: the no-successful-run branch
(warns, floors at the cap, exits 0) and the unset-`GITHUB_WORKFLOW`
branch (recovery skipped, unchanged). `shellcheck` clean; `bash -n`
clean. Also exercised after the review follow-ups: the fetch-limit
truncation warning (via a copy with `RUN_LIMIT=5`) and the anchor-query
failure path (exits 1 rather than degrading). See [the follow-up
comment](#838 (comment))
for the full branch table.

</details>

## Gates

- **Evidence level**: Critical — a clearly wrong outcome (the analysis
records an all-clear for a window it never examined, and the window is
then unreachable), which acts on 1 occurrence. Occurrences: 1 this run,
0 historical (no prior gist entry records an outage of a length that
would trigger it).
- **Structural, not stochastic**: no decision point. The anchor query is
deterministic; replayed ten times it picks the failed run ten times.
- **Change type**: targeted fix — one predicate, one query limit, two
stderr warnings. Normal evidence bar under Gate 2, met.
- **Dedup**: checked open issues #831, #830, #829, #828, #827, #824,
#822, #817, #816, #801, #799, #750, #624 and open PRs #826, #825, #823,
#821, #819, #818, #809. #816 / #818 / #823 / #809 all concern outage
*reporting* — naming the cause in the annotation, the stranded trigger,
per-run comment dedup — none touch window recovery. Searching issues and
PRs for `list-recent-runs` surfaces only the three merged predecessors
#526, #753 and #784.

Evidence log: https://gist.github.com/19b5ab297bb7ac7e1e9a44d595ccde0f

---------

Co-authored-by: tend-agent <270458913+tend-agent@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

claude-behavior Behavioral issues found by review-reviewers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants