Repository navigation
Fix Promotion Review Findings on Local Pass and Backlog Report - #3406
Conversation
The Copilot round on the develop -> main promotion raised these against content merged into develop today. - pr_review.py wait told an unattested feature pull request that an attested local pass covered its head. The note now says no attestation covers the head yet and names the attest step. - pr-review-conduct's "Copilot's absence blocks" heading contradicted the local-pass exception its own bullet states. - backlog_metrics.py counted a fixed subset of labels while the README said open issues by label, and its docstring said review cost per merged pull request reads from a report whose cost figures cover the month rather than the window. It now counts every carried label after the named ones and adds Copilot review runs per merged pull request. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local pass found the new note keyed on attestation alone, while the poll and the verdict read local_cover, which a Copilot refusal on the head also defeats. The note now uses local_cover. It also fixes the count_by_label docstring, which described printing, and splits the long sentence after the retitled pr-review-conduct heading. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The second local pass found no test for an attested head that Copilot refused, the case the local_cover predicate exists for, and a 33-word sentence on a line the retitle touched. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
A recorded local strict-review pass covers head |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
A non-zero assertion also passes on AWAITING_LOCAL_PASS, which would mean the refusal stopped outranking the hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
A recorded local strict-review pass covers head |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved review findings remain in the refused-head messaging, refusal-specific test assertion, and handoff-label documentation.
3 open issues
What changed in this PR
This pull request fixes promotion review messaging and aligns backlog report labels and Copilot review metrics with their documented behavior.
Changes:
- Gates the local-pass wait note on
local_coverand adds coverage for unattested and refused heads. - Reports non-handoff issue labels and Copilot review runs per merged PR.
- Updates documentation, canonical review guidance, generated mirrors, and the source digest.
| File | Summary |
|---|---|
tests/test_pr_review.py |
Tests local-pass messaging and refused-head behavior. [P2 moderate, 4 votes] Assert the refusal-specific exit code 41 rather than merely asserting a non-zero result. |
tests/test_backlog_metrics.py |
Tests additional labels and the Copilot review-runs-per-merged-PR ratio. |
scripts/README.md |
Documents label ordering and calendar-month billing scope. [P2 nit, 2 votes] Clarify that handoff issues are excluded from label output. |
scripts/pr_review.py |
Adjusts the promotion wait note based on local coverage. [P1 moderate, 3 votes] Do not direct an already-attested refused head to publish an attestation; distinguish refusal from a missing attestation or use neutral wording. |
scripts/backlog_metrics.py |
Counts additional non-handoff labels and renders the new review ratio. |
.github/skills/pr-review-conduct/SKILL.md |
Regenerated GitHub skill mirror. |
.claude-plugin/fleet-skills/skills/pr-review-conduct/SKILL.md |
Regenerated Claude skill mirror. |
.claude-plugin/fleet-skills/.source-digests/pr-review-conduct |
Updated generated source digest. |
.agents/skills/pr-review-conduct/SKILL.md |
Clarifies the local-pass exception in the canonical review guidance. |
Review effort: Lite
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #3406 +/- ##
========================================
Coverage 81.60% 81.61%
========================================
Files 19 19
Lines 10231 10235 +4
========================================
+ Hits 8349 8353 +4
Misses 1882 1882
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Copilot on the fix pull request read the shared note as sending an already-attested head to attest. The note now says which of the two defeats the local cover. The README states that the label line also leaves out handoff issues. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
A recorded local strict-review pass covers head |
## Summary Promotes these pull requests from develop. Each carries a maintainer decision recorded on #3391. - [#3399](#3399): states the write guard's threat model and documents two shapes it keeps allowed, a quoted run-time `-C` value and an `xargs` replace string run as a shell script (item 1). - [#3401](#3401): lists a pre-existing review finding in the pull request body instead of filing an issue for it (item 2). - [#3402](#3402): keeps one standing handoff issue per track, edited in place each round, with a `close` subcommand that ends a track (item 5). - [#3400](#3400): adds a weekly backlog and review-cost report script. - [#3403](#3403): runs the Copilot review once per change, at the promotion, with the attested local pass covering a pull request into develop (item 4). - [#3404](#3404): aligns the remaining review rule text with the promotion-only Copilot review (items 2 and 4). - [#3406](#3406): fixes the findings this promotion's first Copilot round raised on the content above, in the `wait` note for a head no local pass covers, the pr-review-conduct heading, and the backlog report's label line and cost scoping. Closes #3382 Closes #3385 ## Pre-existing findings, not filed - The [Copilot round on 4a77a1d](#3405 (review)) headline says documentation does not consistently describe current-head Copilot refusal handling. `local_cover` in `scripts/pr_review.py` refuses an attested head whose current Copilot round is a refusal, and `wait` exits `41` there. The `wait` entry of the `pr_review.py` docstring ("An attested head is covered at once.") and the `scripts/README.md` paragraph on the defined review moments ("Where it is attested, `wait` polls the head's checks instead.") leave that exception out. `pr-review-conduct` Merge Gate item 2 states it ("A refusal is not that coverage"). Both omissions are on `main` already, so this promotion did not introduce them. 🤖 Generated with [Claude Code](https://claude.com/claude-code)


Answers the Copilot round on the develop -> main promotion #3405, whose findings sit on content merged into develop today.
Changes
scripts/pr_review.py:waiton a pull request into a branch other than the default printed "its head is covered by an attested local pass" where no attested local pass covered the head, while the verdict returnedstatus=AWAITING_LOCAL_PASS. The note is gated onlocal_cover, the predicate the poll and the verdict read, so a head with no attestation and an attested head Copilot refused each get a note naming what is still owed. Tests cover both cases and the attested case..agents/skills/pr-review-conduct/SKILL.md: the heading "Copilot's absence blocks, and is answered elsewhere." contradicted the local-pass exception its own bullet states. It reads "Copilot's absence blocks where no local pass stands in, and is answered elsewhere." The two long sentences under it are split. Mirrors regenerated withbuild_dist.py.scripts/backlog_metrics.pyandscripts/README.md: the README says the report lists open issues by label, while the script counted a fixed subset. The named labels still print first, a zero included, and every other label an open non-handoff issue carries follows them, sorted. The docstring said review cost per merged pull request reads from the report, but Actions minutes and the Copilot gross cover the calendar month while the counts cover the window. The report adds aCopilot review runs per merged PR (ratio)column over the window, and the docstring and README state the month scoping.Verification
python3 -m unittest discover -s tests-> OKuvx ruff@latest format --check scripts tests,uvx ruff@latest check scripts tests,uvx mypy@latest-> cleanpython3 scripts/build_dist.py --check-> currentpython3 scripts/prose_lint.py --diff origin/develop-> no violations, and with--check sentence-length-> no violationsattested()and passes withlocal_cover().Three recorded
local-strict-reviewpasses at thefrontiertier. The first raised the note's predicate (attestedrather thanlocal_cover), a docstring that said "print", and a 47-word sentence, each fixed in 34e23c1. The second raised the missing refused-head test and a 33-word sentence, fixed in 2e0d435. The third raised one finding with the edit budget spent, listed below.Open findings from the local pass
introduced:tests/test_pr_review.py,test_an_attested_head_copilot_refused_is_not_called_coveredasserts only a non-zero exit, so a regression from the refusal's exit41toAWAITING_LOCAL_PASS(49) would still pass. Fixed in b05e99a, which asserts exit41and fails with the note gated onattested(). A fourth recorded pass over that push raised nothing.Review
Copilot reviewed 2e0d435 and raised three findings, each fixed and answered in its thread: the shared wait note read as sending a refused, already-attested head to attest (5a202fd), the non-zero exit assertion (b05e99a), and the README label sentence that did not say handoff issues are left out (5a202fd). A fifth recorded pass over 5a202fd raised nothing.
🤖 Generated with Claude Code