feat(create-pr): replace check-conflicts with preflight-check, add wait-for-merge - #578
Merged
Conversation
added 7 commits
March 26, 2026 10:55
…provement Skill for improving a file (prompt, config, template) toward reference quality through repeated single-change A/B testing. Each iteration makes exactly one change, tests it against reference examples via parallel generation + judge evaluation, and adopts only what works. Key design decisions: - One change per iteration (bundled changes hide what helped/hurt) - Dynamic next-change based on judge feedback (not pre-planned list) - Same inputs every iteration for valid comparison - Adopt on TIE (change that doesn't hurt is worth keeping) - Stop after 2+ consecutive rejections (rethink approach)
Replace eval-specific iterative-eval with a general-purpose iterate skill. Works for any incremental improvement: prompts, code, configs, docs — anything where you can verify whether a change helped. Key design: - One change per iteration (clear signal) - Pluggable verification (test, build, judge, user, metric, composite) - Feedback-driven next change (not pre-planned) - Stop on 2+ consecutive rejections BREAKING CHANGE: removes iterative-eval skill
Replaced by preflight-check.sh (Task 2). Tests migrated to test_create_pr_verify_status.bats (Task 1).
baleen37
enabled auto-merge (squash)
March 26, 2026 02:40
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📝 WalkthroughWalkthroughThe create-pr workflow was simplified: the standalone conflict check and verify-pr-status steps were removed and replaced by a blocking preflight-check before push and a wait-for-merge step after PR creation. Documentation, two new scripts, and tests were updated accordingly. Changes
Sequence DiagramsequenceDiagram
participant User as User/Automation
participant Local as Local Repo/Script
participant Origin as Git Origin
participant GitHub as GitHub
participant CI as CI System
User->>Local: run create-pr workflow
Local->>Origin: preflight-check: fetch origin/base
Origin-->>Local: remote state
Local->>Local: check behind/conflicts & query branch protection
alt preflight fails (behind/conflict)
Local-->>User: exit non-zero with advisory
else preflight passes
Local->>Local: commit changes
Local->>Origin: push branch
Origin-->>Local: push OK
Local->>GitHub: create PR + enable auto-merge
GitHub-->>Local: PR created
Local->>GitHub: wait-for-merge: gh pr checks --watch
GitHub->>CI: run/check CI
CI-->>GitHub: checks complete
GitHub-->>Local: PR merged or open/blocked
Local-->>User: finish (success or error)
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/skills/test_create_pr_verify_status.bats (1)
6-9:⚠️ Potential issue | 🟠 MajorStale script references will cause test failures.
The setup references
verify-pr-status.shandsync-with-base.sh, but based on the PR summary, these scripts are being replaced/removed. Lines 11-49 testverify-pr-status.shwhich appears to no longer exist.#!/bin/bash # Check if the referenced scripts exist ls -la plugins/me/skills/create-pr/scripts/🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/skills/test_create_pr_verify_status.bats` around lines 6 - 9, The setup() function exports VERIFY_SCRIPT and SYNC_SCRIPT pointing to verify-pr-status.sh and sync-with-base.sh which are removed/renamed; update setup() to reference the new script paths or remove these exports and the dependent tests (the block that exercises verify-pr-status.sh around lines 11-49) so tests don't fail; specifically edit the setup() exports for VERIFY_SCRIPT and SYNC_SCRIPT (and any usages of VERIFY_SCRIPT in tests) to match the current scripts in plugins/me/skills/create-pr/scripts/ or adjust the tests to the new verification flow after confirming the actual filenames with a directory listing.
🧹 Nitpick comments (2)
tests/skills/test_create_pr_verify_status.bats (2)
76-94: Consider usingTEST_TEMP_DIRfrom bats_helper for consistency.These tests create their own temp directories with manual cleanup, but the loaded
bats_helper.bashalready providesTEST_TEMP_DIRwith automatic cleanup inteardown(). This would ensure cleanup even on test failure.♻️ Optional: use shared temp directory
`@test` "preflight-check.sh: exits 2 when not in a git repository" { PREFLIGHT_SCRIPT="${BATS_TEST_DIRNAME}/../../plugins/me/skills/create-pr/scripts/preflight-check.sh" - TEMP_DIR=$(mktemp -d) - cd "$TEMP_DIR" + cd "$TEST_TEMP_DIR" run env -u GIT_DIR -u GIT_WORK_TREE "$PREFLIGHT_SCRIPT" main [ "$status" -eq 2 ] [[ "$output" =~ "Not in a git repository" ]] - rm -rf "$TEMP_DIR" }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/skills/test_create_pr_verify_status.bats` around lines 76 - 94, Replace the manual temp-dir creation/cleanup in the two tests ("preflight-check.sh: exits 2 when not in a git repository" and "preflight-check.sh: exits 2 when no base branch given and gh fails") by using the shared TEST_TEMP_DIR provided by bats_helper: remove mktemp -d, remove explicit rm -rf cleanup, and cd into "$TEST_TEMP_DIR" before running PREFLIGHT_SCRIPT; ensure bats_helper.bash is sourced so TEST_TEMP_DIR is available and rely on its teardown for automatic cleanup.
129-143: Resource cleanup placement is fragile.Calling
teardown_git_reposbefore assertions means temp directories are cleaned up even when tests pass. However, if the test fails before reachingteardown_git_repos(e.g., on line 139), the directories leak. Consider using bats'teardownfunction for guaranteed cleanup.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/skills/test_create_pr_verify_status.bats` around lines 129 - 143, The test currently calls teardown_git_repos inside the test body (in test "preflight-check.sh: exits 0 when branch is up to date and clean"), which can leak temp repos if the test fails before that call; instead add a bats teardown() function that calls teardown_git_repos so cleanup always runs, and remove the inline teardown_git_repos call from the test body; ensure PREFLIGHT_SCRIPT and setup_git_repos remain in the test but that the final assertions run before any manual cleanup inside the test.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/skills/test_create_pr_verify_status.bats`:
- Around line 104-123: In setup_git_repos, the test assumes the local branch is
named main which fails when Git's default is master; fix by explicitly creating
or pushing to main: when creating the bare repo call git init --bare
--initial-branch=main (if supported) and/or ensure the commit is on a main
branch by running git checkout -b main before the initial commit in
TEST_CLONE_A, and replace the unconditional git push origin main with git push
origin HEAD:main so the current branch is pushed to remote main regardless of
its local name.
---
Outside diff comments:
In `@tests/skills/test_create_pr_verify_status.bats`:
- Around line 6-9: The setup() function exports VERIFY_SCRIPT and SYNC_SCRIPT
pointing to verify-pr-status.sh and sync-with-base.sh which are removed/renamed;
update setup() to reference the new script paths or remove these exports and the
dependent tests (the block that exercises verify-pr-status.sh around lines
11-49) so tests don't fail; specifically edit the setup() exports for
VERIFY_SCRIPT and SYNC_SCRIPT (and any usages of VERIFY_SCRIPT in tests) to
match the current scripts in plugins/me/skills/create-pr/scripts/ or adjust the
tests to the new verification flow after confirming the actual filenames with a
directory listing.
---
Nitpick comments:
In `@tests/skills/test_create_pr_verify_status.bats`:
- Around line 76-94: Replace the manual temp-dir creation/cleanup in the two
tests ("preflight-check.sh: exits 2 when not in a git repository" and
"preflight-check.sh: exits 2 when no base branch given and gh fails") by using
the shared TEST_TEMP_DIR provided by bats_helper: remove mktemp -d, remove
explicit rm -rf cleanup, and cd into "$TEST_TEMP_DIR" before running
PREFLIGHT_SCRIPT; ensure bats_helper.bash is sourced so TEST_TEMP_DIR is
available and rely on its teardown for automatic cleanup.
- Around line 129-143: The test currently calls teardown_git_repos inside the
test body (in test "preflight-check.sh: exits 0 when branch is up to date and
clean"), which can leak temp repos if the test fails before that call; instead
add a bats teardown() function that calls teardown_git_repos so cleanup always
runs, and remove the inline teardown_git_repos call from the test body; ensure
PREFLIGHT_SCRIPT and setup_git_repos remain in the test but that the final
assertions run before any manual cleanup inside the test.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ce0a00e1-93b4-47b4-be93-4d9674494d27
📒 Files selected for processing (6)
plugins/me/skills/create-pr/SKILL.mdplugins/me/skills/create-pr/scripts/check-conflicts.shplugins/me/skills/create-pr/scripts/preflight-check.shplugins/me/skills/create-pr/scripts/wait-for-merge.shtests/skills/test_check_conflicts.batstests/skills/test_create_pr_verify_status.bats
💤 Files with no reviewable changes (2)
- tests/skills/test_check_conflicts.bats
- plugins/me/skills/create-pr/scripts/check-conflicts.sh
added 2 commits
March 26, 2026 11:49
…ibility setup_git_repos() needs user.name/email for git commit to work in CI environments where global git config is not set. Also use -b main for git init --bare to ensure consistent default branch name.
…nd wait-for-merge.sh Replace check-conflicts.sh references with new script names after the create-pr skill improvement.
baleen-release-app Bot
pushed a commit
that referenced
this pull request
Mar 26, 2026
# [16.0.0](v15.0.0...v16.0.0) (2026-03-26) ### Features * **create-pr:** replace check-conflicts with preflight-check, add wait-for-merge ([#578](#578)) ([83f8a5c](83f8a5c)) ### BREAKING CHANGES * **create-pr:** removes iterative-eval skill * test(create-pr): add failing tests for preflight-check and wait-for-merge * feat(create-pr): add preflight-check.sh replacing check-conflicts.sh * feat(create-pr): add wait-for-merge.sh for post-PR merge completion * remove(create-pr): delete check-conflicts.sh and its test file Replaced by preflight-check.sh (Task 2). Tests migrated to test_create_pr_verify_status.bats (Task 1). * feat(create-pr): simplify workflow to 5 steps, use preflight-check and wait-for-merge * fix(create-pr): configure git user in integration tests for CI compatibility setup_git_repos() needs user.name/email for git commit to work in CI environments where global git config is not set. Also use -b main for git init --bare to ensure consistent default branch name. * fix(tests): update me-specific.bats to reference preflight-check.sh and wait-for-merge.sh Replace check-conflicts.sh references with new script names after the create-pr skill improvement.
This was referenced Mar 26, 2026
This was referenced Apr 13, 2026
5 tasks
1 task
This was referenced Jun 22, 2026
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace
check-conflicts.shwith broaderpreflight-check.sh(adds BEHIND detection + branch protection advisory) and addwait-for-merge.shfor post-PR merge completion. Simplifies SKILL.md from 8 steps to 5.Changes
preflight-check.sh: BEHIND + conflict + advisory branch protection checks before pushwait-for-merge.sh: blocks ongh pr checks --watch, handles MERGED/OPEN/CLOSED statescheck-conflicts.shand its dedicated test fileTests
Summary by CodeRabbit
Documentation
Refactor
Tests