Skip to content

The review guard called a fix awaiting release an attack: a merged dev PR that changed claude-review.yml went red on every dev PR as 'review disabled repo-wide'. A third arm reads the base's copy back to the merged PR that produced it and says 'pending release'; the hostile arm is unchanged and a lookup that dies there fails closed - #3673

Merged
erikdarlingdata merged 1 commit into
devfrom
fix/guard-drift-pending
Sep 19, 2026

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

The lie

The Claude review guard has a never-ran arm for the case where claude-code-action refuses to run because .github/workflows/claude-review.yml differs from the default branch (cause (1) in the header). It had two verdicts: if THIS PR touches the file, a warning ("a human must review it", exit 0); otherwise ::error title=Review disabled repo-wide and exit 1. That second verdict was built for one thing -- an unreviewed copy of the file sitting on a branch -- and tonight it fired on every dev PR for something else entirely: #3657, a reviewed and merged dev PR, changed the file, and dev has not been released to main yet. The arm could not tell the two apart because it never looked at where the base branch's copy CAME FROM. It only knew the file differed.

Why this window is a normal state, not an incident

check-pr-branch.yml allows pull requests to main only from dev. So a fix to the review workflow cannot land on main and dev together outside a release: it merges to dev through a reviewed PR, and from that moment until the next dev -> main release every dev PR is skipped by the action. That is the branching model working as designed. A guard that goes red on all of them for the whole window is a mark people learn to ignore -- which the guard's own header names as the failure that costs more than it catches.

The three arms

  1. This PR edits the file -- unchanged. Warning, exit 0, a human reviews it.
  2. Pending release -- NEW. Applies when the base branch's copy of $WF was put there by a merged pull request into the base. The read is provenance, not content: the newest commit on $BASE touching $WF (commits?sha=$BASE&path=$WF), the pull request that introduced that commit (commits/<sha>/pulls, filtered to merged_at != null and base.ref == $BASE, because once a release PR from dev is open the commit is associated with it too), and then three facts held together: it merged into $BASE, its diff includes $WF, and $BASE's blob of $WF equals the blob at its merge commit -- the branch carries exactly what that merge produced and nothing after it. All three hold: ::warning title=Review pending release naming the PR, a summary line, exit 0. The PR is still NOT reviewed and the message says so.
  3. Hostile -- unchanged text, unchanged exit 1. Reached when no merged PR into $BASE introduced the commit (a direct push), when the PR that did does not account for the branch's current copy, or when the base already matches the default branch and nothing visible from here explains the skip. Each of those leaves its own evidence line in the step summary before the existing error.

And a fourth outcome that is not a verdict: drift origin unknown. Every read in the new arm goes through lookup_closed(), a twin of the #2309 lookup() helper that differs ONLY in what a failed call becomes. lookup() warns and exits 0 because a lookup failure is not a review verdict; lookup_closed() errors with ::error title=Drift origin unknown and exits 1, because in this arm passing IS the false verdict -- a hostile drift waved through on a 503. The two share one gh_capture() so the stderr-isolation and mktemp-tolerance fixes from #2309's review live in one place.

Why provenance by commit, not by scanning recent PRs

The obvious shape is to page the most-recently-updated closed PRs into dev and look for one that touched the file. Measured against the live repo two hours after #3657 merged, #3657 was already at index 10 of 30 in that ordering (every PR that gains a comment moves ahead of it), and this repo merges a train a night -- so inside a day the fix falls out of any fixed window and the arm flips back to a false hostile for the rest of the drift window. The commits-with-path read is exact regardless of the branch's age, costs three to five calls instead of up to thirty-one, and commits/<sha>/pulls is documented to return the merged PR that introduced a commit not yet on the default branch. The pulls/<n>/files check is kept as an independent fact so the verdict never rests on the association alone.

Live runs against the repo as it stands tonight

The harness is the guard's run: block extracted with a YAML parser (the same reading Actions does), helpers and the never-ran arm taken by line range and proven identical by diff before each run. R=erikdarlingdata/PerformanceMonitor DEFAULT_BRANCH=main WF=.github/workflows/claude-review.yml PR=3670 (a real dev PR that does not touch the file), never_ran=1 preset. Only the stubbed cases replace one gh api endpoint each; everything else hits the live API.

Case 1 -- live, BASE=dev (dev blob a700b3b9 ≠ main blob 06430dc3; newest dev commit touching the file is 1dae2ade, #3657's squash):

::warning title=Review pending release::Claude cannot run on any PR into dev because .github/workflows/claude-review.yml changed on dev in merged PR #3657 (A Claude review that posted nothing no longer finishes green: read-only tools end the denial churn, the job fails when no verdict was submitted, and the transcript survives the runner) and main has not synced yet. Reviews resume at the next dev -> main release. This PR is NOT Claude-reviewed -- a human must review it.
--- step summary:
### Claude review never ran
- Pending release: `.github/workflows/claude-review.yml` on `dev` is what merged PR #3657 produced, and `main` has not synced. Reviews resume at the next release; a human must review this PR.
--- exit code: 0

Case 2 -- live, BASE=main (base equals default; pending arm skipped, falls to the hostile arm):

::error title=Review disabled repo-wide::Claude never ran and this PR does not touch .github/workflows/claude-review.yml, so that file has drifted from main. Every PR in the repo is silently going unreviewed until the two match.
--- step summary:
### Claude review never ran
- `.github/workflows/claude-review.yml` on `main` already matches `main`; nothing visible from here explains the skip.
- Unexpected: `.github/workflows/claude-review.yml` has drifted from `main`; ALL PRs go unreviewed.
--- exit code: 1

Case 3a -- hostile, wrong PR (stub: commits/<sha>/pulls claims #3668, a real merged dev PR that did NOT touch the file; the files and blob reads are live):

::error title=Review disabled repo-wide::Claude never ran and this PR does not touch .github/workflows/claude-review.yml, so that file has drifted from main. Every PR in the repo is silently going unreviewed until the two match.
--- step summary:
### Claude review never ran
- `.github/workflows/claude-review.yml` on `dev` was last touched by `1dae2adecf08` (PR #3668), but that PR's diff and merge blob do not account for the branch's current copy.
- Unexpected: `.github/workflows/claude-review.yml` has drifted from `main`; ALL PRs go unreviewed.
--- exit code: 1

Case 3b -- hostile, direct push (stub: commits/<sha>/pulls returns []):

::error title=Review disabled repo-wide::Claude never ran and this PR does not touch .github/workflows/claude-review.yml, so that file has drifted from main. Every PR in the repo is silently going unreviewed until the two match.
--- step summary:
### Claude review never ran
- `.github/workflows/claude-review.yml` on `dev` was last touched by `1dae2adecf08`, which no merged pull request into `dev` introduced.
- Unexpected: `.github/workflows/claude-review.yml` has drifted from `main`; ALL PRs go unreviewed.
--- exit code: 1

Case 4 -- fail closed (stub: the commits-with-path call returns HTTP 503):

::error title=Drift origin unknown::gh api (newest dev commit touching .github/workflows/claude-review.yml) failed: gh: HTTP 503: Service Unavailable (...). Claude never ran on this PR, and this guard could not determine whether .github/workflows/claude-review.yml's drift from main is a merged dev change awaiting release or an unreviewed edit. An unexplained drift is the one condition this check exists to make loud, so it fails CLOSED rather than guess. Re-run this guard; if the failure repeats, read .github/workflows/claude-review.yml's history on dev by hand.
--- step summary:
### Claude review never ran
- Drift origin UNKNOWN: lookup failed at newest dev commit touching .github/workflows/claude-review.yml. Failing closed.
--- exit code: 1

Regression -- the refactored fail-OPEN lookup() (stub: pulls/$PR/files returns HTTP 502):

::warning title=Guard lookup failed::gh api (PR changed files) failed: gh: HTTP 502: Bad Gateway. A lookup failure must not become a review verdict (#2309), so the review is UNCONFIRMED -- read the PR's comments to confirm it by eye, or re-run this guard.
--- exit code: 0

actionlint and shellcheck -s bash on the extracted block: both clean.

What it does NOT do

  • It does not touch claude-review.yml. This guard never invokes claude-code-action (header, lines 24–27), so editing it causes no drift refusal anywhere; this PR is itself Claude-reviewable in principle and is only skipped tonight because of the very window it names.
  • It does not relax the hostile arm. Same ::error, same summary line, same exit 1, and it is reached in every case the new arm cannot positively account for -- including the case where the base already matches the default branch and the skip is unexplained.
  • It does not make a pending-release PR "reviewed". The warning says a human must review it, the same words as the edits-the-file arm.
  • It does not change triggers or permissions. contents: read and pull-requests: read were already granted; the contents, commits, and commit-pulls endpoints need nothing more.
  • It does not pass on a failed lookup. That is the one place the [CI] Review guard hard-fails on an unguarded lookup 404 — a lookup failure becoming the verdict its own comments forbid #2309 fail-open rule is deliberately inverted, and the message says why.

No issue exists for this; the PR body is the record.

… never ran and the PR did not touch claude-review.yml, the drift arm had one verdict, "review disabled repo-wide", for both a merged dev PR waiting on the next release and an unreviewed edit nobody read. A third arm reads the base branch's copy back to the merged PR that produced it and says "pending release" instead; the hostile arm is unchanged, and a lookup that dies there fails closed

PRs to main come only from dev (check-pr-branch.yml), so a fix to the review
workflow cannot land on main and dev together outside a release: it merges to
dev through a reviewed PR and every dev PR is skipped by claude-code-action
until the release ships it. #3657 did exactly that tonight and every dev PR
went red for it. The arm now asks provenance, not content: the newest commit
on the base touching the file, the merged PR that introduced it, that PR's
diff including the file, and the base's blob equal to the blob at its merge.
All three hold: warning, exit 0, still a human's review. Any fails, or no PR
claims the commit: the existing exit 1, text unchanged. Every read goes
through lookup_closed(), a fail-CLOSED twin of the #2309 lookup(), because
passing on a 503 would be the false verdict.
@erikdarlingdata
erikdarlingdata merged commit cb30135 into dev Sep 19, 2026
7 of 8 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/guard-drift-pending branch September 19, 2026 01:06
erikdarlingdata added a commit that referenced this pull request Sep 19, 2026
… PRs across the #3653, #3542 and #3691 waves, #3704, #3710, #3712 and #3735, spliced once (#3751)

Sixty-five PRs merged to dev since #3672 (the first splice) across two seats -
the coordinator's #3653 brains-review residue lanes and the wave-2 worker's
#3542 / #3691 PostgreSQL-target engine lanes - and by rule none touched this
file; each returned its entry to a buffer. This is the one splice. Text-only;
one file; nothing outside ## [Unreleased] and the definition block moves.

[Unreleased] gains a ### Changed between ### Added and ### Fixed (Keep-a-Changelog
order) for the nine entries whose subject is a designed behaviour that now behaves
differently - routing (#3712 confidence chooses the channel), gating (#3692 tempdb
persistence, #3734 blocking/LRQ gates, #3724 peak-and-mean), the wire shape
(#3719 one error envelope), the review guard's drift arm (#3673), lineage
metadata (#3723) and two grading changes (#3693, #3747). Twenty-two new
capabilities append to ### Added (the sixteen PostgreSQL-target fact families and
rungs, CONFIG_CHANGED, the recurrence label, the V134/V135 columns, the review
traps file, the measurement census); thirty-two defects append to ### Fixed.
Order within each section is ascending PR number, as #3672 did.

Every entry is one line and verbatim from its buffer (trailing whitespace only).
Four PR pairs fold into one entry as the buffers wrote them (#3738+#3746,
#3683+#3686, #3681+#3685, #3726+#3729); #3695 carries three entries. #3673 has
no issue and cites none, like the file's ninety-six other ref-less entries. Five
definitions added ([#3691], [#3704], [#3710], [#3712], [#3735]) at the end of the
[Unreleased] run, which stays one ascending run (3517..3735). No archive touched;
the split was not run.

tools/changelog/changelog_archive.py verify: all PASS (1,483 bold entries, floor
1,329; 1,363 distinct refs resolve; 1,372 definitions; CRLF throughout; 409,263
bytes under the ceiling). Client-hygiene grep over the file: zero hits.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant