feat(iterate): add generic iterate skill, replace iterative-eval - #567
Conversation
…provement Skill for improving a file (prompt, config, template) toward reference quality through repeated single-change A/B testing. Each iteration makes exactly one change, tests it against reference examples via parallel generation + judge evaluation, and adopts only what works. Key design decisions: - One change per iteration (bundled changes hide what helped/hurt) - Dynamic next-change based on judge feedback (not pre-planned list) - Same inputs every iteration for valid comparison - Adopt on TIE (change that doesn't hurt is worth keeping) - Stop after 2+ consecutive rejections (rethink approach)
Replace eval-specific iterative-eval with a general-purpose iterate skill. Works for any incremental improvement: prompts, code, configs, docs — anything where you can verify whether a change helped. Key design: - One change per iteration (clear signal) - Pluggable verification (test, build, judge, user, metric, composite) - Feedback-driven next change (not pre-planned) - Stop on 2+ consecutive rejections BREAKING CHANGE: removes iterative-eval skill
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe script's post- Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~15 minutes Possibly related PRs
Poem
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
# [9.0.0](v8.0.0...v9.0.0) (2026-03-23) ### Features * **iterate:** add generic iterate skill, replace iterative-eval ([#567](#567)) ([0e8683e](0e8683e)) ### BREAKING CHANGES * **iterate:** removes iterative-eval skill * fix(create-pr): stop auto-merge when verify reports broken state
Summary
Generic iterate skill 추가 및 iterative-eval 대체, create-pr auto-merge broken state 버그 수정.
Changes
Tests
Breaking
Summary by CodeRabbit