docs(eval-harness): fix skill description to follow CSO guidelines

[baleen37]

Summary

Fix eval-harness skill description to comply with writing-skills CSO guidelines — description now contains only triggering conditions, not workflow summary.

Changes

• Removed workflow summary from description ("compare two variants using parallel subagent execution...")
• Simplified variant type enumeration
• Added explicit "Do NOT use" condition

Tests

Manually verified description follows "Use when..." format with no workflow content.

Summary by CodeRabbit

Documentation

• Updated eval-harness documentation to support comparing multiple types of variants including code, prompts, commands, and custom executables with revised input formats and defaults.
• Restructured evaluation phases with improved separation between output collection and grading workflows.
• Enhanced anonymization rules and flexible grading configurations.
• Clarified variant behavior rules and reporting structure.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: aa5d4bd8-f728-4da7-9425-a8a3343f5b93

📥 Commits

Reviewing files that changed from the base of the PR and between 6f37cc8 and 27e3eac.

📒 Files selected for processing (1)

• plugins/me/skills/eval-harness/SKILL.md

---

📝 Walkthrough

Walkthrough

This PR comprehensively rewrites the eval-harness SKILL.md documentation to broaden its scope from code-only comparisons to comparing multiple executable variant types (code, llm, command, custom). The changes introduce an explicit variant-type system, restructure the three-phase execution workflow, update input/output specifications, refine worktree handling to code-only variants, and adjust grading behavior with anonymization rules.

Changes

Cohort / File(s)

Summary

Eval-Harness Documentation plugins/me/skills/eval-harness/SKILL.md

Comprehensive rewrite broadening scope from code to multi-variant execution; introduces variant-type system (code, llm, command, custom); restructures phases with Phase 1 focused on output collection, Phase 2 on auto-grading with anonymization, and Phase 3 on reporting; updates worktree handling to code-only variants; adds new fields (VARIANT_A, VARIANT_B, INPUTS, Evals, Grader) and refines key rules.

Possibly related PRs

• feat(eval-harness): add eval-harness skill for worktree-based code evaluation #554: Directly revises the same eval-harness SKILL.md with changes to variant types, worktree behavior, and phase structure.
• chore(eval): remove eval skill and clean up cross-references #555: Related removal of old eval workflow documentation by deleting plugins/me/skills/eval/SKILL.md.
• feat(skills): add eval-harness skill and eval command #499: Introduced the eval-harness skill that this PR comprehensively revises and extends.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 Whiskers twitching with delight,
Multi-variants now unite!
Code, commands, prompts aligned,
Phases flow, no type confined,
Eval-harness hops more bright! ✨

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch docs/improve-eval-harness-skill-description

📝 Coding Plan

• Generate coding plan for human review comments

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Tip

You can validate your CodeRabbit configuration file in your editor.

If your editor has YAML language server, you can enable auto-completion and validation by adding # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json at the top of your CodeRabbit configuration file.