Skip to content

pr_review.py: a GitHub Actions billing-failure notice can be misread as UNRECOGNIZED_REVIEWER_OUTPUT #1071

Description

@ptr727

What happened

Driving ptr727/Financial-Modeling#199, pr_review.py wait reported:

UNRECOGNIZED REVIEWER OUTPUT (1): ...
  body carrying no heading at all, which no measured review body does  (round 49064e1b)
status=UNRECOGNIZED_REVIEWER_OUTPUT ...

Direct GraphQL confirmed the flagged body:

copilot-pull-request-reviewer, submitted 2026-08-28T23:06:34Z: "The job was not started because
recent GitHub Actions payments have failed or your spending limit needs to be increased."

This isn't a code review at all - it's GitHub Actions' own billing-failure notice (the same one
blocking every workflow job on the repo at that moment, confirmed separately via gh run view's
own annotations), somehow posted as a PR review body attributed to the Copilot reviewer account
rather than surfacing only as a run annotation. The timestamp lines up exactly with the billing
outage window on this repo (recent payment failure / spending limit).

Why this matters

The script's own caution here is correct - it should not trust a digest it cannot fully parse - but
its message doesn't distinguish "an unrecognized review shape" from "not a review at all, an
infra-failure artifact." The two need different next steps: a genuinely new review body shape is a
parser gap to fix; a billing-failure notice masquerading as a review is closer to the class #1060
already tracks (checking budget/quota status before reporting a reviewer bot as broken), just
inverted - here a billing failure got misattributed as reviewer output instead of causing an
absent one.

Suggested fix

Recognize this specific shape (or the general "no code-review markers, matches a known
GitHub-infra-failure phrasing" pattern) and report it distinctly, e.g. status=REVIEWER_INFRA_FAILURE
with a pointer to check the repo's Billing & plans settings, rather than folding it into the generic
UNRECOGNIZED_REVIEWER_OUTPUT bucket that asks the agent to file an issue every time it recurs.

What I did instead

Manually verified via gh api graphql (querying reviews.nodes with commit.oid) that: (1) the
flagged body is the billing artifact above, dated exactly during the confirmed outage window, (2)
the most recent real review from the same reviewer (copilot-pull-request-reviewer, later commit)
reads "Approval recommended... no remaining correctness gaps identified", and (3) reviewThreads
shows 0 unresolved. Proceeded to merge on that basis, per the script's own "whether to merge anyway
is the maintainer's call" - this note is that call, made with the evidence in hand rather than by
overriding the caution unread.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    scriptA defect in hub tooling

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions