Skip to content

Create issue when merged PRs had unresolved comments #875

Open

Description

@strawgate

When the author missed comments or they were outdated or resolved but not actually fixed

Specifically not trying to just replay feedback but real reproducible bugs that's slipped through using the reviews comments and PR comments as the starting context

Activity

  1. github-actions commented on Apr 5, 2026

    @github-actions
    Contributor
    3 related issues

    Note

    🔒 Integrity filter blocked 1 item

    The following item were blocked because they don't meet the GitHub integrity level.

    • #704 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

    To allow these resources, lower min-integrity in your GitHub frontmatter:

    tools:
      github:
        min-integrity: approved  # merged | approved | unapproved | none

    What is this? | From workflow: Trigger Duplicate Issue Detector

    Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

  2. github-actions commented on Apr 5, 2026

    @github-actions
    Contributor

    TL;DR: this is not a duplicate, and the best implementation path is to add a new scheduled detector workflow (or a Bug Hunter variant) that scans recently merged PRs for high-signal unresolved/outdated review feedback, then files an issue only when a concrete, reproducible bug is confirmed.

    Recommendation

    Implement this as a new workflow based on the scheduled-audit + create-issue pattern (not pr-actions-detective). Keep Bug Hunter’s strict reproduction gate: review-thread signals should only be triage input, not automatic issue output.

    Findings
    1. Existing PR-context data already captures thread state, but only for a single PR event/input:

      • .github/workflows/gh-aw-fragments/pr-context.md:6 (PR_NUMBER from event/input)
      • .github/workflows/gh-aw-fragments/pr-context.md:89-97 (review_comments.json, unresolved_threads.json, resolved_threads.json, outdated_threads.json)
    2. The repo already has the right quality bar and issue-output pattern:

      • .github/workflows/gh-aw-bug-hunter.md:75-83 (create-issue, close-older-key)
      • .github/workflows/gh-aw-bug-hunter.md:95-108 (mandatory reproducible bug requirement)
    3. pr-actions-detective is not the right owner for this request:

      • .github/workflows/gh-aw-pr-actions-detective.md:79-82 is scoped to failed PR CI checks
      • .github/workflows/gh-aw-pr-actions-detective.md:64-67 uses noop/PR comment outputs, not issue filing
    4. Related prior work exists and is completed:

      • Issue #497 (closed) and PR #503 added unresolved/resolved/outdated precomputed thread files; this request extends that idea to post-merge bug detection.
    Verification

    I ran targeted repo checks to confirm current behavior and insertion points:

    $ ls .github/workflows | grep -Ei 'merged|unresolved|comment.*detector|review.*detector'
    # (no matches)
    
    $ grep -nE 'merged PR|gh pr list --state merged' .github/workflows/gh-aw-bug-hunter.md
    139:- The bug is already fixed. Before filing, search for recently merged PRs ... `gh pr list --state merged ...`
    
    $ grep -nE 'PR_NUMBER:|review_comments.json|unresolved_threads.json|resolved_threads.json|outdated_threads.json' .github/workflows/gh-aw-fragments/pr-context.md
    6:      PR_NUMBER: "${{ github.event.pull_request.number || inputs.target-pr-number || github.event.issue.number }}"
    89:        | jq -s 'add // []' > /tmp/pr-context/review_comments.json
    93:        > /tmp/pr-context/unresolved_threads.json
    95:        > /tmp/pr-context/resolved_threads.json
    97:        > /tmp/pr-context/outdated_threads.json

    I also confirmed prior-art linkage via GitHub reads:

    • Issue #497 exists and is closed as completed
    • PR #503 is merged and contains the thread-file additions
    Detailed Action Plan
    1. Create a new workflow source, e.g. .github/workflows/gh-aw-merged-pr-review-regression-detector.md, using imports/patterns from:

      • .github/workflows/gh-aw-scheduled-audit.md:1-15 (scheduled audit framing)
      • .github/workflows/gh-aw-fragments/safe-output-scheduled-audit-issue.md:1-8 (create-issue policy)
    2. In the workflow instructions, define candidate collection over recently merged PRs (last N days), then for each PR gather:

      • review threads with isResolved/isOutdated
      • PR review comments and PR discussion comments
      • merge metadata (mergedAt, commit SHA)
    3. Add a strict triage gate in prompt logic mirroring Bug Hunter (.github/workflows/gh-aw-bug-hunter.md:95-108):

      • thread/comment text is only a lead
      • file an issue only when a real, reproducible bug is demonstrated
      • otherwise emit noop
    4. Add duplicate prevention and issue lifecycle controls:

      • title prefix + close-older-key (same pattern as Bug Hunter / Branch Actions Detective)
      • check prior findings before filing (scheduled-audit pattern)
    5. Add user-facing docs/example trigger:

      • gh-agent-workflows/<new-workflow>/README.md
      • gh-agent-workflows/<new-workflow>/example.yml
      • docs page under docs/workflows/gh-agent-workflows/
    6. Recompile generated lock files and run repo validation commands (make compile, make lint) before merging.

    Related Items
    Type Link Relevance
    Issue #875 Current request
    Issue #497 Prior request that added precomputed thread-state files
    PR #503 Merged implementation of unresolved/resolved/outdated thread artifacts
    File .github/workflows/gh-aw-fragments/pr-context.md:6,89-97 Existing thread-state data extraction (single-PR scoped)
    File .github/workflows/gh-aw-bug-hunter.md:75-83,95-108 Reproducible-bug + issue filing quality gate
    File .github/workflows/gh-aw-pr-actions-detective.md:64-67,79-82 Not suitable owner (PR CI-failure comment workflow)
    File .github/workflows/gh-aw-scheduled-audit.md:81-123 Reusable scheduled detector + dedupe pattern

    Note

    🔒 Integrity filter blocked 1 item

    The following item were blocked because they don't meet the GitHub integrity level.

    • #399 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

    To allow these resources, lower min-integrity in your GitHub frontmatter:

    tools:
      github:
        min-integrity: approved  # merged | approved | unapproved | none

    What is this? | From workflow: Trigger Issue Triage

    Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions