Skip to content

release-merge: check_suite re-evaluation never fires, so an approval given before CI is green leaves the release PR stuck forever #318

Closed

Description

@Shinrai

What's happening

workflow-release-merge.yml (#303/#305) is documented to work even when the maintainer approves the release PR before CI finishes: the approval fires pull_request_review: submitted, and if any check is still pending it exits cleanly and waits for a check_suite: completed event to re-evaluate. In practice, once approval genuinely precedes CI going green, that re-evaluation never happens — the PR sits approved and fully green with nothing left to merge it.

Evidence (CLDMV/slothlet PR #437)

  • Approved at 2026-09-21T03:38:29Z. merge / 🚦 Release Merge ran immediately (pull_request_review, run 35558209246), completed successfully at 03:38:41, but did not merge — PR Coverage Badge (needs ci to finish first, in this repo's currently-released @v4) was still running.
  • PR Coverage Badge finished at 03:57:04; ✅ Required PR Check went green at 03:57:16. The PR has been fully green and approved since then.
  • No further run of release-merge.yml occurred after 03:38. Querying its workflow-run history for check_suite-triggered runs turns up none for this PR at all — every historical check_suite-triggered run of this workflow reports head_branch: master, never next, going back well before the approval.
  • The PR only unstuck after a fresh pull_request_review was submitted, which re-ran the evaluation from scratch and merged immediately (everything was already green).

This is likely the first real exercise of the check_suite re-fire path: every slothlet release before #437 merged because the approval came after CI was already green, so the initial pull_request_review evaluation succeeded on the spot and the re-evaluation path was never actually needed.

Root cause (unconfirmed — needs a repro with logging)

workflow-release-merge.yml's caller (release-merge.yml) listens on check_suite: types: [completed] with no further filtering, and action.mjs's resolveReleasePr() correctly handles a check_suite event by resolving head_branch (github.event.check_suite.head_branch) to the open release PR. The logic looks right — the gap appears to be upstream of the job even running: GitHub does not appear to be delivering a check_suite: completed event to this workflow for the release PR's own head branch (next/hotfixes) at all. Possible causes to check:

  • The check suite for the release PR's head SHA may be getting merged/attributed differently than assumed (e.g. multiple check suites per SHA across different workflow-triggering events, and the "last" one completing doesn't fire the webhook the way expected).
  • A repository or org-level check-suite auto_trigger_checks preference for the Actions app.
  • check_suite webhooks are known to have caveats when a SHA is associated with more than one ref — worth checking whether the release PR's head SHA is being treated as ambiguous.

Suggested fix

Whatever the exact platform cause, the fix needs to not depend on check_suite alone:

  1. Add a fallback poll. A schedule (e.g. every 5–10 minutes) or a repeated re-arm from the ci workflow itself (e.g. have ci.yml's own completion, on the release PR's head, dispatch workflow_dispatch on release-merge.yml) so an approval-before-green PR is guaranteed to converge even if check_suite doesn't fire.
  2. Alternatively/additionally, have the CI workflow chain (workflow-ci.yml or the reusable it composes) explicitly call the release-merge workflow via workflow_call/workflow_dispatch as its last step when it detects it's running on a release PR head with a standing approval, rather than relying on GitHub's own check-suite webhook timing.
  3. At minimum, document the gap and the manual recovery (workflow_dispatch on release-merge.yml, or re-submitting any review) so a stuck PR isn't mysterious.

Impact

Affects every repo that has opted into release-merge.yml (slothlet at least; .github is being wired up now — see #316) whenever a maintainer approves a release PR before its CI (in particular the coverage leg, which can run long) finishes. The PR silently sits fully green and approved with nothing to merge it — easy to mistake for "just needs one more click" rather than a stuck automation.

Activity

  1. added
    type: bugSomething is broken or not behaving as expected
    area: coreTouches core library / runtime source code
    type: ciChanges to CI workflows, actions, or build pipelines
    on Sep 21, 2026
  2. added theissue type on Sep 21, 2026
  3. added
    priority: highNeeds attention soon — not blocking, but don't let it sit
    status: implementedBuilt and deployed, but not yet fully tested/verified
    on Sep 28, 2026
  4. Shinrai commented on Sep 28, 2026

    @Shinrai
    ContributorAuthor

    Fix shipped in v4.29.0 via #319 (a61d9e8): the re-evaluation trigger is now workflow_run: completed on the caller's own CI workflow instead of check_suite, which GitHub never delivers for Actions-created suites. workflow-release-merge.yml reads workflow_run.head_branch first, and both local-release-merge.yml and the release-flow-v4/release-merge.yml template carry the new trigger.

    Leaving this open at status: implemented. The workflow_run runs do fire (slothlet has 100+ of them since), but every release PR since then — slothlet #440, #444, #457, #467 — was approved after CI was already green and merged straight from the pull_request_review run. The approve-before-green path this issue is about hasn't run live yet. The next release approved while CI is still running will confirm it; close then.

    Note for adopters: the trigger lives in each repo's own release-merge.yml, so a consumer only gets it by re-syncing that file from the template. @v4 alone doesn't bring it.

  5. Shinrai commented on Sep 28, 2026

    @Shinrai
    ContributorAuthor

    The first live approve-before-green case, #322 (v4.29.1), got stuck, so the workflow_run fix in #319 was incomplete.

    • Approved at 05:00:03. CI finished at 05:00:10 and re-fired release-merge (run 36380024924). That run logged waiting on 1 unfinished check(s): 🔍 Analyze javascript-typescript and exited.
    • CodeQL finished at 05:00:51. It isn't in workflow_run.workflows (only 🧪 Local CI is), so nothing re-fired.
    • The gate waits on every check on the head commit, from any workflow, but only CI's completion re-armed it. Whichever workflow finishes last has to be listed, and CodeQL routinely outlasts CI.
    • The approval's own run (release: v3.1.2 #12) shows as cancelled. A later workflow_run run replaced it in the concurrency group's pending slot. That part is harmless: every run re-reads the approval and the checks from the API.

    Follow-up fix on fix/release-merge-rearm-all-checks (5fdce27): list every workflow that puts a check on a release-PR head, in both local-release-merge.yml and the caller template. Also add branches: [next, hotfixes] so feature-branch CI no longer triggers it.

    Every repo that adopted the template carries the same one-entry list and needs to re-sync its release-merge.yml: slothlet, slothlet-vine, vitest-runner, jsonv, jsonv-eslint-plugin-jsonv, kasa-api, rti-videowall-server, tv-control, unifi-cli.

  6. added
    status: in progressPartially implemented — actively being built
    and removed
    status: implementedBuilt and deployed, but not yet fully tested/verified
    on Sep 28, 2026
  7. cldmv-bot commented on Sep 28, 2026

    @cldmv-bot
    Contributor

    Closed in v4.29.2 — resolved by #323.

  8. Shinrai commented on Sep 28, 2026

    @Shinrai
    ContributorAuthor

    Re-synced every adopter's release-merge.yml from the v4.29.2 template, keeping each repo's own header. All nine had the template body unchanged, and the new list covers every workflow that puts a check on their release PRs.

    • PRs into next: slothlet#490, slothlet-vine#45, vitest-runner#49, jsonv#61, jsonv-eslint-plugin-jsonv#40
    • Direct to dev (dev-trunk repos): io-kasa-api e960d1e, rti-videowall-server 51e390c, tv-control 6d0d730, unifi-cli 64752b5. These take effect once dev is promoted to master.

    Each change takes effect in that repo once it reaches master. That repo's first release PR after that is the live test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: coreTouches core library / runtime source codepriority: highNeeds attention soon — not blocking, but don't let it sitstatus: in progressPartially implemented — actively being builttype: bugSomething is broken or not behaving as expectedtype: ciChanges to CI workflows, actions, or build pipelines

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions