Skip to content

Count Copilot Findings Under the Unresolved Bullet and File Table - #3642

Merged
ptr727 merged 6 commits into
developfrom
feature/auto-3610
Oct 30, 2026
Merged

ptr727 merged 6 commits into
developfrom
feature/auto-3610

Conversation

@ptr727

@ptr727 ptr727 commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

Summary

A Copilot ccr-overview-v2 round can open on "Approval recommended" and state "No open issues". Its narrative can still name findings in a nested "Review findings remain unresolved:" list and in the file table's Findings column. The reader counted none of them. So the digest printed CLEAN VERDICT HEADLINE for a round it said counts no finding, and wait exited 0 with nothing pointing at the findings.

  • The nested label is now a findings-list spelling in scripts/reviewers/copilot.json.
  • The file table's Findings column is read through a new table_findings vocabulary key. One table_cells reader replaces table_paths, so the paths and the column come from a single walk.
  • A new Reading.listed field holds the larger of the list count and the filled Findings cells, since both name the same findings. unthreaded takes it into account, so FINDINGS WITH NO THREAD (2) prints for the body in Count the Findings Under Copilot's "Review Findings Remain Unresolved" Bullet in pr_review.py #3610.
  • Reading.open stays the count the round states. So overview=T/M and its ? reading keep their meaning for the Merge Gate.
  • A clean round whose list or table names a finding no longer prints CLEAN VERDICT HEADLINE. A flagging round in the same shape still exits 50 as it did on develop.
  • No exit code changes. The documented contract keeps FINDINGS WITH NO THREAD advisory.

Closes on promotion: #3610

Verification

  • tests/test_pr_review_copilot.py and tests/test_pr_review.py gain constructed cases for the Count the Findings Under Copilot's "Review Findings Remain Unresolved" Bullet in pr_review.py #3610 shape, the list alone, the column alone, a stated count beside the lists, and a flagging verdict that still exits 50. Each new case fails against the base reader.
  • The whole suite passes (python -m unittest discover -s tests), tests/test_pr_review_size.py included.
  • ruff format, ruff check, mypy, the prose gate, build_dist.py --check, and the eol gate pass.

Local pass findings

The first push's spent edit budget left four introduced findings open. The next push fixed each:

  • scripts/pr_review_digest.py dropped the headline under a flagging verdict carrying a findings list. Fixed in 872b4ed, which also added the test assertion for it.
  • scripts/pr_review.py --help said the headline alone names what the round flags. Fixed in 872b4ed.
  • Lines this change edited ran past the wrap width. Rewrapped in 872b4ed.
  • The flagging wording named only the findings list, while listed can come from the file table alone. The pr-review-conduct skill still called the headline the one place. Fixed in dd213c6.

A later pass raised one more introduced finding, also fixed:

  • scripts/pr_review_copilot.py the Reading docstring said listed is the findings the Findings column names, while a cell counts once however many it names. Fixed in fc8eb49, which states the possible undercount. The pass over fc8eb49 raised no finding.

🤖 Generated with Claude Code

ptr727 and others added 3 commits October 29, 2026 22:47
A Copilot overview round can open on "Approval recommended" and state
"No open issues" while its narrative names findings in a nested
"Review findings remain unresolved:" list and in the file table's
Findings column. The reader counted none of them, so the digest
printed CLEAN VERDICT HEADLINE as a round counting no finding.

The nested label is now a findings-list spelling, and the file table's
Findings column is read through a new `table_findings` vocabulary key.
Where the list or the column names more findings than the round
states, their count wins and reaches `open`, so the round reads as one
stating that many open findings. An unbelievable stated count still
reads as None.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ding

The listed findings move to their own `listed` field rather than
raising `open`, so `overview=T/M` still reads the count the round
states and a round stating none still reads `?`. A clean round whose
lists name a finding no longer prints CLEAN VERDICT HEADLINE as one
counting none, while a flagging round still exits 50 as it did.

One table reader now returns the header labels and body cells, so the
file table's paths and its findings column come from a single walk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flagging round that states no open finding still exits 50 when its
narrative lists findings, but its digest and status lines no longer
say it counts no finding. They name the headline and the findings
list together. The docs now say a clean round whose findings list or
file table names a finding prints no CLEAN VERDICT HEADLINE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings October 30, 2026 06:02
@ptr727

ptr727 commented Oct 30, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head 2b7304d2e0bde2e727c3ccd97fd39675c9799c58, the content this pull request carries at that commit against develop, and it recorded 4 findings.

@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The flagging branch for a round whose narrative lists findings dropped
the headline line, so exit 50 printed nothing to answer. Both branches
now print it, and the test asserts that. The --help text for exit 50
names the findings list beside the headline, and the lines this change
edited are rewrapped to the surrounding width.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Several active findings remain, including a moderate issue that drops the flagging headline from output and a missing test assertion, along with documentation, status-message, and formatting inconsistencies.

6 open issues
What changed in this PR

This pull request updates Copilot review parsing so findings named in unresolved lists or file-table Findings columns are counted and surfaced without changing existing exit codes.

Changes:

  • Adds listed finding tracking and unified table-cell parsing.
  • Updates verdict handling, digest output, reviewer vocabularies, and documentation.
  • Adds regression coverage for list-only, table-only, clean, flagging, and stated-count scenarios.
  • Synchronizes the review-conduct skill distributions.

This pull request requires changes.

File Summary
tests/​test_pr_review.py Adds end-to-end coverage for listed findings and verdict output.
tests/​test_pr_review_copilot.py Tests nested finding lists, Findings columns, and stated counts.
scripts/​reviewers/​qodo.json Adds the shared table_findings vocabulary key.
scripts/​reviewers/​copilot.json Adds unresolved-list and Findings-column vocabulary.
scripts/​reviewers/​coderabbit.json Adds the shared table_findings vocabulary key.
scripts/​README.md Documents list and file-table finding sources.
scripts/​pr_review.py Updates status help and verdict messaging.
scripts/​pr_review_rounds.py Accounts for listed findings in verdict classification.
scripts/​pr_review_digest.py Renders advisory output for unthreaded listed findings.
scripts/​pr_review_copilot.py Parses finding lists and file-table cells into Reading.listed.
.github/​skills/​pr-review-conduct/​SKILL.md Updates generated review-conduct guidance.
.claude-plugin/​fleet-skills/​skills/​pr-review-conduct/​SKILL.md Updates the Claude skill distribution.
.claude-plugin/​fleet-skills/​.source-digests/​pr-review-conduct Refreshes the skill source digest.
.agents/​skills/​pr-review-conduct/​SKILL.md Updates the canonical review-conduct guidance.

Review effort: Lite

Comment thread scripts/pr_review_digest.py Resolved
Comment thread tests/test_pr_review.py Resolved
Comment thread scripts/pr_review.py Resolved
Comment thread scripts/pr_review.py Resolved
Comment thread scripts/pr_review.py Resolved
Comment thread scripts/pr_review_rounds.py Resolved
A round's listed findings can come from the file table's findings
column alone, so the flagging-verdict lines now name it beside the
findings list. The pr-review-conduct skill no longer calls the
headline the one place such a round names what it flags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.70%. Comparing base (b38e084) to head (fc8eb49).
⚠️ Report is 20 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3642      +/-   ##
===========================================
+ Coverage    81.60%   81.70%   +0.09%     
===========================================
  Files           26       26              
  Lines         9070     9119      +49     
===========================================
+ Hits          7402     7451      +49     
  Misses        1668     1668              
Flag Coverage Δ
python-3.13 81.70% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ptr727

ptr727 commented Oct 30, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head dd213c6f6ea9dbe09a7b423a4a3097211e2f55ea, the content this pull request carries at that commit against develop, and it recorded 1 finding.

The `Reading` docstring said `listed` is the findings the file table's
findings column names. It counts each filled cell once however many
findings that cell names, so the docstring now says the table alone
can undercount.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ptr727

ptr727 commented Oct 30, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head fc8eb493ba022579577ec213c9360c8077c2a59e, the content this pull request carries at that commit against develop, and it recorded 0 findings.

@ptr727
ptr727 merged commit 8beff32 into develop Oct 30, 2026
13 checks passed
@ptr727
ptr727 deleted the feature/auto-3610 branch October 30, 2026 06:28
ptr727 added a commit that referenced this pull request Oct 30, 2026
Answers the Copilot finding raised on promotion #3667 against
`scripts/pr_review_copilot.py`, which #3642 introduced.

- **A Copilot Findings cell now counts each finding it names.** The file
table's Findings column counted one finding per filled cell, so a second
finding in the same cell went uncounted. On a clean-verdict round that
leaves it out of triage. A new per-reviewer vocabulary pattern,
`table_finding_marker`, is `\[P\d\]` for Copilot and empty for
CodeRabbit and Qodo. A filled cell counts each marker it carries, and
once where it carries none, so no count drops below what it was before.
- The `Reading` docstring states the new counting.

## Verification

- The full suite passes, with 3123 tests run and 7 skipped.
- The new subtest, a cell carrying two markers beside a cell carrying
none, counts 3. It fails against develop's adapter, which counts 2.
- ruff, mypy, the diff-scoped prose gate, and the eol gate are clean.

## Dropped findings

- `scripts/pr_review_copilot.py`: a `[Pn]` written inside a code span or
in prose about another finding also counts. The only effect is a higher
count in the digest's no-thread line, a loud failure rather than a
silent one, so it is left as is under "Trust Boundaries and Hardening
Effort".

Closes on promotion: none. The finding was raised on #3667 and has no
issue of its own.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
ptr727 added a commit that referenced this pull request Oct 30, 2026
…Session Prompt Rules (#3667)

Promotes the changes merged into `develop` since promotion #3636 to
`main`.

- [#3640](#3640) Pins the
validator's `uvx ruff` to a Dependabot-managed manifest.
- [#3642](#3642) Counts
Copilot findings under the unresolved bullet and the file table.
- [#3644](#3644) Aligns
the split comment copies and drops the stale counts.
- [#3646](#3646) Sends a
command meant for the session's input line in a plain message, never in
a blocking prompt.
- [#3648](#3648) Opens the
attended session by cloning a hub checkout of the session's own.
- [#3650](#3650) Reads a
held package as installed in both Linux host scripts.
- [#3652](#3652) States
the PowerShell `-Help` and `-h` form in the command-line argument rules.
- [#3654](#3654) Counts
sentences in Markdown table cells for the sentence-length check.
- [#3658](#3658) Splits a
table row only at an unescaped pipe in the semicolon rule.
- [#3660](#3660) Finds
table cell boundaries before blanking code spans and quotations.
- [#3662](#3662) Names a
stale job record on a completed run in `pr_review.py`.
- [#3665](#3665) Keeps a
line holding code after a block comment's closer out of `comment-added`.
Its squash commit on `develop` carries the pull request's original
title, which names a heredoc reading. That reading was withdrawn before
the merge on decision #3666, so a workflow `run:` heredoc body still
reads as code.
- [#3669](#3669) Counts
each `[Pn]` marker in a Copilot Findings cell, so a second finding in
one cell is no longer dropped from triage. It answers one of the two
findings Copilot raised on this promotion. The other is declined with a
parser probe in its thread.

Closes #3616
Closes #3610
Closes #3622
Closes #3638
Closes #3604
Closes #3511
Closes #3602
Closes #3456
Closes #3655
Closes #3657
Closes #3362
Closes #1664

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants