Skip to content

Place a Copilot Outstanding Findings List and Read a Table Leaving Out Only Lineless Files as Covering - #3702

Merged
ptr727 merged 5 commits into
developfrom
feature/burndown-3672-3489
Oct 31, 2026
Merged

ptr727 merged 5 commits into
developfrom
feature/burndown-3672-3489

Conversation

@ptr727

@ptr727 ptr727 commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • pr_review.py Leaves a Copilot Overview's Outstanding Findings List Unplaced #3672, a Copilot findings list it noted rather than placed.
    • scripts/reviewers/copilot.json gains the outstanding findings: and unresolved review findings: spellings under labels and finding_lists. This is a vocabulary change with no new code branch.
    • The reader now reads numbered findings lists as well as bulleted ones. Each item keeps its text and the path:line ranges its code spans name.
    • A span counts only where its path is a changed file or a thread's path, so node:20 is not read as a line.
    • FINDINGS WITH NO THREAD prints every item with a mark: [thread at its line], [no thread at its line], [outdated thread on its path], [thread page cut, unknown], or [no path:line code span].
    • An item whose span no thread sits on now counts toward that block's number, even where the open count and the linked entries balance.
  • Read a Copilot File Table That Leaves Out Only Binary Files as Covering Them #3489, a file table leaving out only binary files.
    • The changed-file query now reads additions, deletions, and changeType.
    • A file added, modified, or deleted with no added or deleted line may go unnamed in the table, which then reads coverage=table. A rename, a copy, a type change, and a file missing a count are never left out.
    • Nothing is left out unless a file with an added or deleted line remains. The reason line says how many files were left out.
  • Read a Thread's Outdated Flag Rather Than a Null Line in pr_review_copilot.py's listed_lines #3701, a current file-level thread read as outdated. Copilot's first round raised it on this branch. Q_FULL now reads each thread's isOutdated flag, so a null line no longer stands for outdated. A file-level thread marks no item, and the item stays counted.
  • scripts/pr_review.py --help and scripts/README.md describe these changes.

How a Binary File Is Known

GitHub reports additions=0 and deletions=0 for a binary file. Its changed-file list carries no binary flag, and neither the GraphQL nor the REST shape adds one.

A constructed case: a pull request changes docs/a.md (3 added lines) and adds static/b.png (0/0). Copilot's table names docs/a.md alone. The digest now reads coverage=table with the reason "names exactly the 1 changed file(s), 1 more with no changed line left out". Before this change it read coverage=none.

Two alternatives were not taken. Guessing from the extension misses an unusual binary and wrongly excuses a text file with a binary-looking name. Reading .gitattributes would mean a content fetch per pull request.

New Exit Path

A findings-list label followed by no list the reader can read now leaves a note rather than being placed silently. An example is Unresolved findings: followed by None.. At the top level of a body, that note is also an unplaced part, so status exits 50 until its fingerprint is acknowledged with comment --fingerprint. Inside a collapsed section it is a note only.

Remaining Findings

  • The maintainer's call on the Read a Copilot File Table That Leaves Out Only Binary Files as Covering Them #3489 trade. An empty file, a mode-only change such as dropping an executable bit, and a text file that .gitattributes marks binary or -diff all read as 0/0. GitHub's file list does not tell them apart from a binary file. Each one therefore may go unnamed in a Copilot table, and the table still reads coverage=table. No reviewer or diff showed the .gitattributes case. The README names this residual. Whether excusing these is acceptable is open.

Local Review Findings

Four local strict-review passes ran. The first two raised findings that were fixed on this branch. The third raised the #3701 finding, which is fixed here after Copilot raised it too. The fourth, on the fix, raised none.

Closes on promotion: #3672, #3489, #3701

🤖 Generated with Claude Code

…t of the Table Match

Add the outstanding and unresolved-review findings-list spellings to the Copilot vocabulary, keep each list item's text, and print it under FINDINGS WITH NO THREAD marked by whether a thread opens at the path and line it names. Read each changed file's additions and deletions, and leave a file GitHub counts no changed line in, such as a binary one, out of the file table comparison where the table does not name it.
…ed Findings From Code Spans

Read changeType and leave out no file unless one with a changed line remains. Read a numbered findings list, and note a findings-list label with no list. Take an item's path:line ranges from its code spans, match them against each thread's range or its original one, mark them unknown on a cut thread page, and count an item no thread sits on toward FINDINGS WITH NO THREAD.
…ather Than Count It

A findings-list span counts as a path and line only where its path is a changed file or a thread's. An outdated thread on an item's path is named rather than matched on its original lines. A file table leaves nothing out unless a file with an added or deleted line remains.
Copilot AI lite review requested due to automatic review settings October 31, 2026 16:05
@ptr727

ptr727 commented Oct 31, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head 3d0a8dd406d2e67e2fe59895952289e461dcf4b4, the content this pull request carries at that commit against develop, and it recorded 1 finding.

@coderabbitai

coderabbitai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical bug misclassifies current file-level review comments as outdated, allowing findings to bypass triage and making verification of the review digest unreliable.

1 open issue
What changed in this PR

This PR expands Copilot review parsing to recognize outstanding findings lists, report findings without matching threads, and allow eligible lineless files to be omitted from file-table coverage.

Changes:

  • Adds findings-list vocabulary, numbered-list parsing, code-span locations, and per-item thread status markers.
  • Supports omitting added, modified, or deleted files with zero additions and deletions from coverage tables.
  • Updates GraphQL queries, digest output, documentation, fixtures, and regression tests.

Review findings:

  • Critical (2 votes): scripts/pr_review_copilot.py:745 infers outdated status from null line fields, which can suppress unthreaded file-level findings. The thread’s explicit outdated flag should be used instead.
  • Moderate (1 vote): scripts/pr_review_copilot.py:721 may attach nested-list content and spans to the parent finding.
  • Nit (1 vote): scripts/README.md:234 uses incorrect subject–verb agreement.
File Description
tests/​test_pr_review.py Tests lineless-file coverage and digest behavior for findings lists.
tests/​test_pr_review_copilot.py Tests findings-list parsing, span matching, thread status, and file omission rules.
scripts/​reviewers/​copilot.json Adds the new Copilot findings-list spellings.
scripts/​README.md Documents findings-list reporting and lineless-file coverage behavior.
scripts/​pr_review.py Requests thread ranges and changed-file metadata from GraphQL.
scripts/​pr_review_rounds.py Applies lineless-file exceptions during table coverage matching.
scripts/​pr_review_digest.py Prints marked findings without matching threads in the digest.
scripts/​pr_review_copilot.py Parses findings-list items, matches code spans to threads, and identifies eligible omitted files.

Review effort: Lite

Comment thread scripts/pr_review_copilot.py Resolved
@codecov

codecov Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.02%. Comparing base (b38e084) to head (9d5fdcf).
⚠️ Report is 42 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3702      +/-   ##
===========================================
+ Coverage    81.60%   82.02%   +0.41%     
===========================================
  Files           26       27       +1     
  Lines         9070     9273     +203     
===========================================
+ Hits          7402     7606     +204     
+ Misses        1668     1667       -1     
Flag Coverage Δ
python-3.13 82.02% <100.00%> (+0.41%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…Listed Findings

A current file-level thread also carries a null line, so it read as outdated and dropped the items citing its file out of the count. Read isOutdated in the full query instead.
@ptr727

ptr727 commented Oct 31, 2026

Copy link
Copy Markdown
Owner Author

A recorded local strict-review pass covers head 9d5fdcff9f04ebf9ffc84397d73e00c57767c39d, the content this pull request carries at that commit against develop, and it recorded 0 findings.

@ptr727

ptr727 commented Oct 31, 2026

Copy link
Copy Markdown
Owner Author

Answers to the two findings in Copilot's overview on 3d0a8dd that carry no thread.

  • Moderate, scripts/pr_review_copilot.py:721, nested-list content and spans attached to the parent finding: declined. A list item at the margin is the finding, and its nested content is part of that finding. entries_in already reads an open section the same way, where a back-reference nested in an entry is not an entry of its own. A constructed case: an item citing src/a.py:10 with a nested line citing src/b.py:40, and one current thread at src/b.py:40. It reads the same as the two spans written on the item's own line. The printed line shows the span that matched, and a mark is a pointer rather than a match. Splitting nested items into findings of their own would inflate listed and unthreaded. The rule that any span may match predates this change.
  • Nit, scripts/README.md:234, subject-verb agreement: fixed in 9d5fdcf. The sentence now reads "an empty file, a mode-only change, and a text file .gitattributes marks binary or -diff all read the same".

@ptr727
ptr727 merged commit b30ce55 into develop Oct 31, 2026
13 checks passed
@ptr727
ptr727 deleted the feature/burndown-3672-3489 branch October 31, 2026 16:29
ptr727 added a commit that referenced this pull request Oct 31, 2026
…ne Revision, and Place Copilot's Outstanding Findings (#3703)

Promotes the changes merged into `develop` since promotion #3693 to
`main`, the second round of a backlog burndown.

- [#3698](#3698) Has
`drive-pr` read `state=` in every digest `wait` or `status` prints. On a
feature pull request a peer merged, the drive skips the merge and goes
on to the post-merge path. On a promotion it reports the merge as the
drive's end state. `state=CLOSED` is a stop. `state=` decides over any
exit code on such a pull request. `scripts/README.md` documents the
`state=`, `merged_at=`, `merge_commit=` and `closed_at=` fields.
- [#3696](#3696) Has
`hub_fetch_run.py` read the prose gate's companion and script twice from
`main` when no hub commit resolves, refusing with "the gate did not run"
when the reads differ, so a hub push between the two requests cannot
pair revisions. A resolved commit is still fetched once.
- [#3702](#3702) Places a
Copilot overview's `Outstanding findings:` or `Unresolved review
findings:` list and prints each item under FINDINGS WITH NO THREAD,
marked by whether a current thread sits at its code-span `path:line`. A
findings label with no readable list holds as an unplaced part. A
changed file with no changed line, added, modified or deleted, is left
out of the table match while a file with changed lines remains, so a
table omitting only binary files reads as covering.

Remaining for the maintainer, from #3702: an empty file, a mode-only
change, and a text file `.gitattributes` marks `binary` or `-diff` read
0/0 like a binary file, so a Copilot table leaving one out still reads
as covering. That is the trade #3489 asked for.

Deferred review findings: #3697 (open findings on a peer-merged feature
pull request get no outcome before cleanup in `drive-pr`) and #3700 (the
three documents describing `hub_fetch_run.py`'s fallback without its new
refusal).

Fixes #3688
Fixes #3533
Fixes #3672
Fixes #3489
Fixes #3701

🤖 Generated with [Claude Code](https://claude.com/claude-code)
ptr727 added a commit that referenced this pull request Nov 2, 2026
… merge Reads (#3793)

## Summary

`pr_review.py merge` now reads an acknowledgment record for findings no
thread carries. It also ends every run on one `next=` line, and the
skills follow that field instead of listing every status line. The
digest gives the excluded-files refusal its own `refusal=EXCLUDED`
value.

### Acknowledgment record (#3718, and the verdict case from #3724)

- New library module `scripts/pr_review_answers.py`. It makes no call
and imports neither the gate nor a transport. It gives each answerable
item a key. That is a SHA-256 prefix over the item's kind, the round or
comment raising it, and the item's text. The items are:
- each `suppressed`, `previously_missed`, and `cr_outside_diff` section
on every round
  - each open Qodo entry
  - the head round's `FINDINGS WITH NO THREAD` block
  - each `REVIEWER OUTPUT NOTED` part
  - a verdict naming no counted finding

One key covers every finding a collapsed section or block holds. A key
holding a finding still awaiting the maintainer takes no `--ack`. A Qodo
entry's key covers its title and text, since Qodo edits one comment in
place. A reviewer login drift gets no key and takes no answer. `merge`
refuses it as `REVIEWER_LOGIN_DRIFT` for the maintainer, since that
login's rounds stay unread while the drift stands.
- `pr_review.py comment --ack KEY=OUTCOME`, repeatable, posts the
written outcomes with one `<!-- fleet-finding-ack: key=<key>
outcome=<outcome> -->` line per item. Some bodies post nothing and exit
`69`: one with no prose outside code (for `--fingerprint` too), one
naming a key no item carries, and one with an outcome the item does not
take. Both acknowledgment kinds count only where markdown-it finds prose
outside code.
- The outcomes are `fixed`, `declined`, `filed`, `listed`, and
`dropped`, after `pr-review-conduct`'s verdicts. `Awaiting maintainer`
has no outcome, so a value call stays the maintainer's.
- A record counts only when it is all of these:
  - from an owner, member, or collaborator
  - in a comment that is not hidden
  - written beside outcomes in prose, not just the marker
  - outside code
- `merge` clears a count only once every item under it carries an
outcome. A partial answer, a new round raising the same text (new key),
a `?` count, or a cut window still refuses.
- A verdict naming no counted finding
(`VERDICT_NAMES_UNCOUNTED_FINDINGS`, exit `50`) clears once its key is
acknowledged. That is the case that blocked #3764 and #3765. It stays
fail-closed:
  - An unanswered verdict still exits `50`.
  - `--ack` refuses `fixed` for a verdict, since a fix moves the head.
- A headline naming a concrete failing input is a real finding under
outcome 1, so the driver fixes it and pushes.
- After a fix push, a head hold that was acknowledged turns into noted
output. That output now reads as answered by its own round's shape
acknowledgment. `shape_hold` takes the commit to read for this. A
refusal's parts never read as answered this way. The shape
acknowledgment and the finding acknowledgment now read a written outcome
the same way, through `answering_comments`.
- `report_verdict` moved from `pr_review.py` into the new module. That
keeps `pr_review.py` under its ceiling (2196 lines before, 2166 now).
The gate-item reading moved out of `pr_merge.py` as well (227 lines
now). No ceiling was raised. The new module has its own ceiling, 450,
and sits in the readers' import contract test (384 lines).

### `next=` route (#3724)

- `merge` passes its output through and ends on
`next=done|cleanup|readback|redrive|retry|maintainer`. One table in
`pr_review_answers.py` decides it from each `status=` line, `wait`'s
gate lines included. A line the table does not name goes to the
maintainer.
- A run naming several routes takes the first of `readback`, `cleanup`,
`redrive`, `retry`, and `maintainer`, so what the seat can clear goes
first. The skill keeps the one-rerun bound.
- A generic `REVIEW_IS_A_REFUSAL` routes to the maintainer, since only a
file-count refusal has a remedy this seat can apply, and the line does
not say which kind it is.
- A test pins every status line `pr_merge.py`, `pr_review.py`, and
`pr_review_answers.py` print to its route.
- `drive-pr` step 4 now follows `next=` and drops its per-status list.
`scripts/README.md` "`pr_merge.py`" carries the route table.
- The Qodo open item: an open Qodo entry is now answered by `--ack` and
needs no `/review` re-review for `merge` to read. The untracked
re-review wait therefore no longer gates a merge.

### Excluded-files refusal kind (#3777)

- The digest prints `refusal=EXCLUDED` for the excluded-files refusal,
apart from `refusal=YES`.
- `scripts/README.md` now lists `excluded` among the adapter's refusal
kinds. The `wait` exit-`41` paragraph now covers the `49`/`0`/`41` split
for that kind and the earlier-head quota or error refusal under an
attested later head. The `coverage=` paragraph already described the
lineless-file allowance (from #3702), so it is unchanged.

### Skills

- `pr-review-conduct` Merge Gate item 3 (verdict) and item 4 (noted
parts), and "Answering a suppressed finding" (the `--ack` outcome
mapping), agree with the code. The generated trees are regenerated.

## Verification

- `tests/test_pr_review.py`, `tests/test_pr_merge.py`,
`tests/test_pr_review_copilot.py`, and `tests/test_pr_review_size.py`
pass, as do ruff, mypy, the prose gate (with the `comments` label), and
`build_dist.py --check`.
- New cases cover:
  - the clearing of each count
- an untrusted, hidden, marker-only, unknown-outcome, or code-quoted
record not clearing
  - a partial answer still refusing
  - per-round keys
  - shape-acknowledged noted parts
- a verdict clearing as `declined` but not as `fixed` or from an
outsider
  - `comment --ack` posting the marker, and refusing with `69`
  - `refusal=EXCLUDED`
- each route, through `merge` and as a table pinned to the status lines
in the sources

Closes on promotion: #3718, #3724, #3777

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants