Repository navigation
Ask an After-Checks Side Reviewer Once CI Settles, and Compare Ask Times in UTC - #3888
Conversation
…mes in UTC An explicit after-checks CodeRabbit or Qodo read the head's checks once, after the Copilot poll, so a head whose CI outlasted Copilot's round was never asked. Where the wait itself polls, the ask now polls the checks with await_green on the wait's own clock, bounded by --timeout. A wait that stops without polling still reads them once. side_asked_on_head compared a comment's UTC createdAt with a commit's offset-bearing committedDate as text. Both are now parsed with utc_instant. Also documents the DIRTY and unreadable-rollup holds at exit 44, drops the PAYLOAD_KEY alias for pr_review_payload.POLICY_KEY, and moves the post-and-confirm of a conversation comment into one post_comment helper that both the gate's comment command and the side asks call. Closes #3885 Closes #3887 Refs #3877 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local pass over the first commit found four defects in it. - The digest and exit code came from a payload read before the after-checks poll, so a check that went green during it could still read as stuck. A wait that polled now re-reads the payload for the digest. - A held head's own poll already waits out the checks, so it no longer hands the side ask a poll. Exit 46 stays immediate. - A pull request merged while its checks settled could still be asked. The re-read after the check wait now checks the state again. - The exit 0 text still said a covered head never polls the checks. Refs #3885 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit stopped a held head from handing the side ask a poll, but the usage text and the README still said any polling wait awaits the checks. They now name a poll for a Copilot round, and say every other wait, a held head's included, reads the checks once. Refs #3885 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
|
A recorded local strict-review pass covers head |
There was a problem hiding this comment.
🟡 Changes recommended
An active P2 finding remains because a sentence in scripts/pr_review_side.py is ungrammatical and obscures the meaning about preserving the committer’s offset.
1 open issue
What changed in this PR
This pull request updates policy-driven side-review timing, UTC timestamp comparisons, comment confirmation, documentation, and related test coverage.
Changes:
- Polls checks after Copilot waits before requesting eligible side reviewers.
- Compares commit and comment timestamps as UTC instants.
- Centralizes comment posting and confirmation, removes the duplicate policy-key alias, and documents exit-44 conditions.
- Adds coverage for polling, closure, timestamp offsets, holds, and comment confirmation.
- Review findings include one unresolved polling issue and two documentation nits.
This pull request requires changes.
| File | Summary |
|---|---|
tests/test_pr_review.py |
Updates tests to use the canonical policy payload key. |
tests/test_pr_review_policy.py |
Adds coverage for timing, polling, closure, UTC comparisons, holds, and comment confirmation. |
scripts/README.md |
Documents after-checks polling and exit-44 conditions. |
scripts/pr_review.py |
Wires side-review polling, fresh payload reads, and shared comment posting into the wait flow. |
scripts/pr_review_side.py |
Implements check polling, UTC timestamp handling, and shared comment confirmation. |
scripts/pr_review_policy.py |
Removes the duplicate payload-key alias. |
scripts/pr_review_payload.py |
Clarifies ownership of the canonical policy-key constant. |
Review effort: Lite
The side_asked_on_head docstring dropped the noun its possessive stood for. Refs #3885 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
A recorded local strict-review pass covers head |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #3888 +/- ##
===========================================
+ Coverage 81.60% 84.13% +2.52%
===========================================
Files 26 32 +6
Lines 9070 10580 +1510
===========================================
+ Hits 7402 8901 +1499
- Misses 1668 1679 +11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
…t Policy (#3892) ## Summary Promotes #3881, #3884, #3886, #3888, #3889, and #3891 to `main`. Closes #3200. Closes #3885. Closes #3887. - **Per-repository review policy (#3881).** `scripts/pr_review.py` reads which bots review a repository, on which bases, when, and how often, from the repository's own policy, with the hub default reproducing today's behavior. - **Review return on cost (#3886).** `scripts/backlog_metrics.py` measures reviewer cost per useful finding. It is now a uv script, so the metrics server's weekly timer must run it as `uv run --no-project --script scripts/backlog_metrics.py` from this release on. - **After-checks side reviewer (#3888).** An explicit after-checks CodeRabbit or Qodo is asked once CI settles, and ask times compare in UTC. - **Commit in place for an operational repository (#3889).** The policy file became `.github/agent-policy.json`, with its schema under `review` and a `guard` section that lets `gh-write-guard` allow an in-place commit on a listed branch. - **Agent policy rendered from the registry (#3891).** `repo-config/configure.sh` renders `.github/agent-policy.json` from `registry/repos.json` and applies the bot settings it implies: the Copilot ruleset rule, CodeRabbit's auto-review keys, and Qodo's auto-feedback key. Once this is on `main`, `configure.sh check` reports agent-policy drift on every repository until `apply --checkout` has been run there. - **Dependabot (#3884).** The actions-deps group bump. Each feature pull request carried recorded local strict review passes, with every finding fixed, declined with evidence, or filed (#3890). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Why
#3881 left two defects in
scripts/pr_review_side.py, and #3885 adds two design items and a documentation gap. Each sits on the review-cutover track (#3877), and the pilot needs the first one fixed before any repository gives CodeRabbit or Qodowhen: after-checks.What
side.askread the head's checks once, after the Copilot poll. So where a Copilot round already covered the head, or landed first, a check still running gavePENDINGand only a note. Nothing asked later, since the merge gate'srun_waitpassesmay_request=False. Now a wait that polls for a Copilot round hands the ask theawait_greenpartial, which polls the checks on the wait's own clock and is bounded by--timeout. Every other wait reads them once. That covers the checks-hold, unrecorded-request, quota and refusal stops, exit 49, and a held head, whose own poll already waits on the checks. A wait that polled re-reads the payload before its digest and exit code, so a check that went green during the side poll is not judged stuck from a stale read. A pull request merged or closed while its checks settled is asked nothing.side_asked_on_headcompared a comment's UTCcreatedAtwith the head'scommittedDateas text, and aGitTimestampkeeps the committer's offset. Both now go throughpr_review_checks.utc_instant.waitusage text andscripts/README.md. The exit-0 text now also names the after-checks exception to "does not poll the checks".PAYLOAD_KEYis gone (Ask an After-Checks Side Reviewer Once CI Settles, and Document the Exit 44 Holds #3885 item 3). Callers readpr_review_payload.POLICY_KEY, and the test asserting the two agree is dropped.side.post_commentposts a conversation comment and says whether its URL and exact body confirm it.comment_on_prand the side asks both call it.scripts/pr_review.pysits at 2199 of its 2200-line ceiling after this change. The next change that adds logic there has to move some first, as #3880 plans.Refs #3885, #3887, #3877. The closing keywords belong on the develop -> main promotion PR.
Verification
uv run --no-project --with-requirements tests/requirements.txt python -m unittest discover -s tests, ran 3539 tests with 11 skipped, all passing, at 6bd9a5c. Thetest_pr_review*suites pass at the head, 709 tests.TestSideAskTimingcases intests/test_pr_review_policy.pywere each run against the pre-change scripts or with their fix reverted, and each one failed. Those are the outlasting-CI ask, the poll hand-off per wait path, the merged-during-poll ask, the fresh digest read, and both UTC offsets.Local-review findings, disposed
Three recorded
local-strict-reviewpasses at thefrontiertier covered the branch. The third, at 08cb68a, raised nothing.introduced, fixed in bc8e6ae: the digest and exit code came from a payload read before the side poll, so a check that went green during it could read asNOT_PICKED_UPand exit 44.introduced, fixed in bc8e6ae: a held head handed the ask a poll, which could delay exit 46 by up to--timeout.introduced, fixed in bc8e6ae: a pull request merged while its checks settled could still be asked.introduced, fixed in bc8e6ae: the usage text and the README0row still said a covered head's exit 0 never polls the checks.introduced, fixed in 08cb68a: the usage text and README still said any polling wait awaits the checks after the held-head change.🤖 Generated with Claude Code