feat: add auto-merge workflow quality checkpoints and troubleshooting guide#48
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces comprehensive documentation and automated checks for GitHub auto-merge workflows, specifically targeting Dependabot and Renovate PRs. It adds a new checklist and pitfall guide to SKILL.md, a canonical workflow template to auto-merge-guide.md, and several new validation checkpoints in checkpoints.yaml. Feedback focuses on improving the security and robustness of the provided workflow template by adding harden-runner and error handling, ensuring consistency in workflow file naming across the automated checks, and expanding the bot detection logic to include release-please bots.
Move verbose auto-merge checklist and pitfalls table to a pointer referencing references/auto-merge-guide.md for canonical content.
There was a problem hiding this comment.
Pull request overview
Adds quality checkpoints and updated guidance for setting up/troubleshooting auto-merge for dependency bot PRs in the github-project skill.
Changes:
- Added mechanical checkpoints GH-23–GH-27 and LLM review checkpoints GH-28–GH-29 focused on auto-merge workflow correctness.
- Added a “Canonical Auto-merge Workflow Template” section to the auto-merge reference guide.
- Added a short “Auto-merge Setup for New Repos” section to
SKILL.md.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| skills/github-project/references/auto-merge-guide.md | Adds a canonical workflow template and design-decision notes for auto-merging dependency PRs. |
| skills/github-project/checkpoints.yaml | Introduces new mechanical + LLM checkpoints to validate auto-merge workflow presence and correctness. |
| skills/github-project/SKILL.md | Adds a new quick-start auto-merge setup section near the troubleshooting content. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| target: .github/workflows/auto-merge-deps.yml | ||
| pattern: 'on:\s*\n\s*pull_request_target:' | ||
| severity: error | ||
| desc: "Auto-merge workflow must use pull_request_target trigger (not pull_request) for bot PR write permissions" | ||
|
|
||
| - id: GH-25 | ||
| type: regex | ||
| target: .github/workflows/auto-merge-deps.yml | ||
| pattern: 'github\.event\.pull_request\.user\.login' | ||
| severity: warning | ||
| desc: "Auto-merge should check github.event.pull_request.user.login (not github.actor which changes on reruns)" | ||
|
|
||
| - id: GH-26 | ||
| type: regex | ||
| target: .github/workflows/auto-merge-deps.yml | ||
| pattern: '--auto' | ||
| severity: warning | ||
| desc: "Auto-merge should use gh pr merge --auto (not direct merge) to respect branch protection and merge queues" | ||
|
|
||
| - id: GH-27 | ||
| type: regex | ||
| target: .github/workflows/auto-merge-deps.yml |
There was a problem hiding this comment.
GH-23 allows either .github/workflows/auto-merge-deps.yml or auto-merge.yml, but GH-24–GH-27 only scan auto-merge-deps.yml. Repos using auto-merge.yml will bypass these quality checks. Consider either (a) narrowing GH-23 to only auto-merge-deps.yml, or (b) duplicating GH-24–GH-27 for auto-merge.yml / using a glob target (e.g., auto-merge*.yml) so the checks apply consistently.
| target: .github/workflows/auto-merge-deps.yml | |
| pattern: 'on:\s*\n\s*pull_request_target:' | |
| severity: error | |
| desc: "Auto-merge workflow must use pull_request_target trigger (not pull_request) for bot PR write permissions" | |
| - id: GH-25 | |
| type: regex | |
| target: .github/workflows/auto-merge-deps.yml | |
| pattern: 'github\.event\.pull_request\.user\.login' | |
| severity: warning | |
| desc: "Auto-merge should check github.event.pull_request.user.login (not github.actor which changes on reruns)" | |
| - id: GH-26 | |
| type: regex | |
| target: .github/workflows/auto-merge-deps.yml | |
| pattern: '--auto' | |
| severity: warning | |
| desc: "Auto-merge should use gh pr merge --auto (not direct merge) to respect branch protection and merge queues" | |
| - id: GH-27 | |
| type: regex | |
| target: .github/workflows/auto-merge-deps.yml | |
| target: .github/workflows/auto-merge*.yml | |
| pattern: 'on:\s*\n\s*pull_request_target:' | |
| severity: error | |
| desc: "Auto-merge workflow must use pull_request_target trigger (not pull_request) for bot PR write permissions" | |
| - id: GH-25 | |
| type: regex | |
| target: .github/workflows/auto-merge*.yml | |
| pattern: 'github\.event\.pull_request\.user\.login' | |
| severity: warning | |
| desc: "Auto-merge should check github.event.pull_request.user.login (not github.actor which changes on reruns)" | |
| - id: GH-26 | |
| type: regex | |
| target: .github/workflows/auto-merge*.yml | |
| pattern: '--auto' | |
| severity: warning | |
| desc: "Auto-merge should use gh pr merge --auto (not direct merge) to respect branch protection and merge queues" | |
| - id: GH-27 | |
| type: regex | |
| target: .github/workflows/auto-merge*.yml |
| ### Auto-merge Setup for New Repos | ||
|
|
||
| Every repo with Dependabot/Renovate needs auto-merge. Key requirements: | ||
| - Enable `allow_auto_merge` on repo | ||
| - Use `pull_request_target` trigger (not `pull_request`) | ||
| - Check `user.login` (not `github.actor`) | ||
| - Use `gh pr merge --auto` with dynamic strategy | ||
|
|
||
| See `references/auto-merge-guide.md` for the canonical workflow and common pitfalls. |
There was a problem hiding this comment.
This change adds an “Auto-merge Setup for New Repos” section, but the PR description says to add an “Auto-merge Checklist for New Repos” plus a “Common Auto-merge Pitfalls” table placed after the existing “Auto-merge Not Working” section. Either update the PR description or add the missing/relocated content so the documentation matches what the PR claims.
| ## Canonical Auto-merge Workflow Template | ||
|
|
||
| ```yaml | ||
| name: Auto-merge dependency PRs | ||
|
|
||
| on: | ||
| pull_request_target: | ||
| types: [opened, synchronize, reopened] | ||
|
|
||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| auto-merge: | ||
| runs-on: ubuntu-latest | ||
| if: >- | ||
| github.event.pull_request.user.login == 'dependabot[bot]' || | ||
| github.event.pull_request.user.login == 'renovate[bot]' | ||
| steps: | ||
| - name: Approve PR | ||
| env: | ||
| PR_URL: ${{ github.event.pull_request.html_url }} | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: gh pr review --approve "$PR_URL" | ||
|
|
||
| - name: Enable auto-merge | ||
| env: | ||
| PR_URL: ${{ github.event.pull_request.html_url }} | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| REPO: ${{ github.repository }} | ||
| run: | | ||
| STRATEGY=$(gh api "repos/$REPO" --jq ' | ||
| if .allow_squash_merge then "--squash" | ||
| elif .allow_merge_commit then "--merge" | ||
| elif .allow_rebase_merge then "--rebase" | ||
| else "--merge" end') | ||
| gh pr merge --auto "$STRATEGY" "$PR_URL" | ||
| ``` | ||
|
|
||
| ### Key Design Decisions | ||
|
|
||
| - **`pull_request_target`**: Required for bot PRs — `pull_request` runs with read-only tokens for fork-like contexts | ||
| - **`user.login`**: Immutable PR author field — `github.actor` changes when humans re-run workflows | ||
| - **`--auto`**: Respects branch protection, merge queues, and required checks — direct merge bypasses these | ||
| - **Dynamic strategy**: Repos may only allow specific merge methods — hardcoding breaks when config changes |
There was a problem hiding this comment.
The “Canonical Auto-merge Workflow Template” omits the step-security/harden-runner step, but GH-28 explicitly asks reviewers to check for harden-runner presence, and the existing template in references/dependency-management.md includes it. To avoid conflicting guidance, either add harden-runner to this canonical template or explicitly call it out as optional and adjust GH-28 wording accordingly.
Summary
checkpoints.yamlcovering auto-merge workflow existence, correctpull_request_targettrigger, bot detection viauser.login,--autoflag usage, and dynamic merge strategy detectionauthor_associationmisconfiguration detectionSKILL.mdwith "Auto-merge Checklist for New Repos" and "Common Auto-merge Pitfalls" table after the existing "Auto-merge Not Working" sectionauto-merge-deps.ymlworkflow template with key design decision rationale toreferences/auto-merge-guide.mdTest plan
checkpoints.yamlYAML is valid (python3 -c "import yaml; yaml.safe_load(open('skills/github-project/checkpoints.yaml'))")