Skip to content

PR review agent has been silently skipping reviews for months #1972

Closed
#1974
Closed
PR review agent has been silently skipping reviews for months#1972
#1974
Assignees
v1vcopilot-swe-agent

Description

@v1v
v1v
opened on Aug 31, 2026

PR review agent has been silently skipping reviews for months — found the root cause + fix
  Been chasing a startup_failure on docs-builder's PR review workflow, which led me to a bigger issue in elastic/ai-github-actions.
  What's broken: The AI PR reviewer occasionally posts a no-op instead of actually reviewing:
  > "No review submitted: required helper command ready_to_code_review is unavailable in this environment"
  Root cause: Our workflow fragments define custom helper tools (ready_to_code_review, ready_to_make_pr, ready_to_push_to_pr) using a safe-inputs: config key. That key was renamed to
  mcp-scripts: upstream in github/gh-aw a while back, and we never migrated. The current compiler doesn't recognize safe-inputs: anymore, so it just silently drops it — the tool never gets registered,
  even though the prompt still tells the agent to call it. It's not a flake, it's a guaranteed failure every time the agent reaches that step.
  This isn't new either — bots have auto-filed and auto-closed this exact finding at least 4 times (#621, #1360, #1802, #1803) over the past several months without anyone actually fixing it, since the
  closes were just "issue expired," not "issue resolved."
  The fix:

Rename safe-inputs: → mcp-scripts: in the 3 affected fragments (same schema, just the key name)
Bump the gh-aw compiler pin v0.87.5 → v0.87.10 for other reliability fixes
Recompile all 57 workflow lock files
Verified: the tool now actually shows up in the compiled MCP server list, make lint + tests pass
  Blocker: I put this together on a fork (Mpdreamz/ai-github-actions@chore/upgrade-gh-aw-v0.87.10), but .github/workflows/** in that repo is locked down — external/unsigned contributors can't open a
  PR touching those paths (confirmed by #1803, where even their own bot hit the same wall). Need someone with real elastic org write access to either push the branch or open the PR from it.
  Compare link: main...Mpdreamz:ai-github-actions:chore/upgrade-gh-aw-v0.87.10

Activity

  1. github-actions commented on Aug 31, 2026

    @github-actions
    github-actionsBot
    on Aug 31, 2026 – with GitHub Actions
    Contributor

    This issue looks highly related to #1803, but it is not safe to close as a duplicate yet because this report includes credible recurrence/regression evidence after prior closures.

    Possible duplicates:

    2 related issues

    What is this? | From workflow: Trigger Duplicate Issue Detector

    Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

  2. v1v commented on Aug 31, 2026

    @v1v
    v1v
    on Aug 31, 2026
    MemberAuthor

    /plan

  3. github-actions commented on Aug 31, 2026

    @github-actions
    github-actionsBot
    on Aug 31, 2026 – with GitHub Actions
    Contributor

    Recommendation: Adopt the upstream key migration (safe-inputs → mcp-scripts) for the PR-review helper fragment, bump the gh-aw compiler pin to the already-validated v0.87.10, then recompile and lint. This is the lowest-risk fix because it aligns source with current compiler behavior and preserves existing review flow (ready_to_code_review writes review guidance files used later in the prompt).

    Findings (evidence):

    1. .github/workflows/gh-aw-fragments/safe-output-code-review.md:2 defines ready-to-code-review under safe-inputs: (legacy key), so current compilers can silently drop registration.
    2. .github/workflows/gh-aw-pr-review.md:134 requires calling ready_to_code_review; if not registered, review flow cannot execute as authored.
    3. .github/workflows/gh-aw-mention-in-pr.md:165 and .github/workflows/gh-aw-mention-in-pr-no-sandbox.md:158 also require ready_to_code_review, so impact is broader than one workflow.
    4. Makefile:4-6 currently pins GH_AW_VERSION, GH_AW_BUILD_VERSION, and GH_AW_COMPAT_VERSION to v0.87.5; issue PR review agent has been silently skipping reviews for months #1972 reports validated fix behavior on v0.87.10 plus successful compile/lint.
    5. Related history confirms recurrence without durable remediation: [prompt-audit] Compiled prompts call undeclared ready_to_make_pr / ready_to_push_to_pr tools #621, [prompt-audit] PR review prompts require unavailable ready_to_code_review tool #1360, Fix create-PR prompts to match available safe-output tools #1803, and operational symptom [aw] PR Review produced no safe outputs #1951.

    Verification:

    • I validated the current source-level mismatch by reading the workflow and fragment files above and confirming the required helper call is present in prompts while helper definition uses the legacy key.
    • I did not rerun compile/lint in this planning pass because PR review agent has been silently skipping reviews for months #1972 already includes a concrete validated branch and outcomes; the implementation plan below includes exact commands to rerun in-repo after patching.
    Detailed Action Plan

    1. Update helper schema key in fragment

      • Edit .github/workflows/gh-aw-fragments/safe-output-code-review.md:
        • Change top-level safe-inputs: to mcp-scripts:.
        • Keep existing ready-to-code-review script content unchanged.

    2. Align compiler version pins

      • Edit Makefile:
        • Set GH_AW_VERSION := v0.87.10
        • Set GH_AW_BUILD_VERSION := v0.87.10
        • Set GH_AW_COMPAT_VERSION := v0.87.10

    3. Regenerate compiled workflows

      • Run:
        • make compile
      • Expect lockfile updates under .github/workflows/*.lock.yml.

    4. Validate repository checks

      • Run:
        • make lint
        • (Optional but prudent if touching prompt behavior broadly) make test

    5. Confirm the specific regression is fixed in generated output

      • Search compiled workflows for PR-review surfaces:
        • Ensure the generated MCP/safe-output configuration includes the helper registration path expected by current compiler behavior.
        • Ensure prompt references to ready_to_code_review remain consistent with available helper tooling.

    6. Open follow-up hardening issue (recommended)

      • Add a small CI/prompt-audit check that fails when prompts require helper commands that are not declared/registered in generated workflow tool surfaces. This prevents repeat silent drift.

    7. Closeout sequencing

      • Land schema migration + version bump + recompilation in one PR to avoid intermediate broken states.
      • After merge, monitor next gh-aw-pr-review runs for absence of “No Safe Outputs Generated” failures tied to missing helper tools.

    Related items: #621, #1360, #1803, #1951; key files: .github/workflows/gh-aw-pr-review.md, .github/workflows/gh-aw-mention-in-pr.md, .github/workflows/gh-aw-mention-in-pr-no-sandbox.md, .github/workflows/gh-aw-fragments/safe-output-code-review.md, Makefile.

    What is this? | From workflow: Trigger Plan

    Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

  4. v1v
    assigned
    Copilot
    and
    v1v
    on Aug 31, 2026
  5. Copilot
    linked a pull request that will close this issueFix silently dropped PR-review MCP tool: rename safe-inputs → mcp-scripts, bump gh-aw to v0.87.10 #1974on Aug 31, 2026

  6. v1v commented on Sep 1, 2026

    @v1v
    v1v
    on Sep 1, 2026
    MemberAuthor

    github/gh-aw#20115 is the one that introduced the chagne in the signature

  7. v1v
    closed this as completedin #1974on Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions