Skip to content

refactor(me): simplify write-skill and split SkillOpt into a reference - #753

Merged
baleen37 merged 2 commits into
mainfrom
feat/improve-verify
Aug 4, 2026
Merged

refactor(me): simplify write-skill and split SkillOpt into a reference#753
baleen37 merged 2 commits into
mainfrom
feat/improve-verify

Conversation

@baleen37

@baleen37 baleen37 commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Summary

write-skill was 205 lines of dense prose with no examples, 12 sections (two of
them both named "Common mistakes"), and it covered only SkillOpt operation
despite being named write-skill.

It is now general skill-authoring guidance in 94 lines / 4 sections, with
SkillOpt operation moved to references/skillopt.md so it costs nothing when
unused.

Changes

  • Rewrite SKILL.md: 205 → 94 lines (~2674 → ~1038 tokens), 12 → 4 sections.
  • Add references/skillopt.md (249 lines) for SkillOpt operation and its traps.
  • Delete agents/openai.yaml — nothing references it, Codex manifests are
    generated per plugin by scripts/generate-codex-plugin-manifests.sh, and
    docs/superpowers/plans/2026-07-18-story-loop-skills.md explicitly forbids
    adding it.
  • Replace SKILL.md prose assertions in tests/me/me-specific.bats with a
    structural check, per the CLAUDE.md rule against asserting skill content.
  • Update the plugins/me/README.md entry, which described the old scope.

Corrected facts

Checked against the runner and its source, not from memory:

  • OpenCode is not a supported source. The repo ships an OpenClaw reference
    adaptation, a different thing. Removed the target path and the flag warning.
  • Cursor was missing entirely — source, backend, and its currently disabled
    tool-aware replay.
  • --scope invoked matches parent directories. _project_matches() accepts
    a session when either path prefixes the other, so ~ and ~/dev sessions
    leak into a repo-scoped harvest; sibling worktrees under other parents are
    unreachable. Only an explicit projects path list in config is correct.
  • --max-sessions is a global cap, not per project. With 3 paths and
    limit=2 it returned 2 sessions from 1 project and dropped the rest silently.
  • An existing last_harvest checkpoint takes precedence over
    --lookback-hours
    , so 0 does not rescan history on an already-harvested
    path.
  • harvest --output plus per-task provenance tags is the only way to merge
    Claude and Codex into one holdout split; transcript_source is file-level, so
    concatenating loses the provider.
  • Also documented: protected SLOW_UPDATE/APPENDIX regions, real spend under
    dry-run, and rollback from the backup adopt writes.

Tests

bats tests/ — 70 pass, 0 fail. pre-commit clean on all changed files.

Verified by invoking the skill in real claude -p sessions, not just subagents:

  • Authoring: produced a 96-line single-file skill with a well-formed
    description and zero SkillOpt leakage.
  • A/B: against no-skill and against a 193-line intermediate draft. The
    shorter version drove tighter output (65 vs 99 lines) and was the only arm
    that ran a scored baseline. Notably the no-skill baseline was already strong,
    which is why the surviving sections target what it actually missed.
  • Regression: the primary authoring path still works after the SkillOpt fix
    below.

One bug only real sessions caught

Subagent trigger simulation scored 6/6, but in a live session the model
announced it would read references/skillopt.md, then answered from memory and
invented flags — every verified fact absent. Replacing the passive
"See references/..." with an explicit prohibition plus the three silent failure
modes fixed it: 0 → 5 correct fact mentions, with the parent-directory leak,
--source auto stopping early, and provenance loss all stated correctly.

Note

The installed me:write-skill 17.41.1 plugin cache still wins skill selection
locally until this releases. Per the known stale-cache behavior, marketplace update alone may not pick it up — a reinstall may be needed.

Summary by CodeRabbit

  • New Features
    • Expanded the skill-writing guidance to cover creating, improving, and tuning agent skills.
    • Added comprehensive guidance for SkillOpt and SkillOpt-Sleep workflows, including validation, safeguards, rollback, and reporting.
  • Documentation
    • Updated the workflow description to include writing or fixing SKILL.md files and baseline validation.
  • Tests
    • Updated checks for the revised documentation and removed obsolete agent metadata validation.

Test User added 2 commits August 4, 2026 15:47
SKILL.md was 205 lines of dense prose with no examples, 12 sections
(including two named "Common mistakes"), and it covered only SkillOpt
operation despite being named write-skill.

Rewrite it as general skill-authoring guidance in 94 lines / 4 sections,
and move SkillOpt operation to references/skillopt.md so it costs nothing
when unused.

Verified by invoking the skill in real `claude` sessions:

- Authoring produced a 96-line single-file skill with no SkillOpt leakage.
- A/B against no-skill and against the 193-line draft showed the shorter
  version drove tighter output and was the only one that ran a scored
  baseline.
- The SkillOpt path initially failed: the model announced it would read
  the reference, then answered from memory and invented flags. Replacing
  the passive "See references/..." with an explicit prohibition plus the
  three silent failure modes fixed it (0 -> 5 correct fact mentions).

Correct facts checked against the runner and its source:

- OpenCode is not a supported source; the repo ships OpenClaw. Removed.
- Add Cursor source/backend and its disabled tool-aware replay.
- `--scope invoked` matches parent directories, so `~` sessions leak in,
  and sibling worktrees under other parents are unreachable. Only an
  explicit `projects` path list in config is correct.
- `--max-sessions` is a global cap, not per project; it silently drops
  whole projects.
- An existing `last_harvest` checkpoint takes precedence over
  `--lookback-hours`, so `0` does not rescan history on a known path.
- `harvest --output` plus per-task provenance tags is the only way to
  merge Claude and Codex into one holdout split.
- Document protected regions, real spend under `dry-run`, and rollback
  from the backup `adopt` writes.

Also drop agents/openai.yaml: nothing references it, Codex manifests are
generated per plugin by scripts/generate-codex-plugin-manifests.sh, and
the project plan explicitly forbids adding it.

Replace the SKILL.md prose assertions in tests with a structural check,
per the CLAUDE.md rule against asserting skill content.
@baleen37
baleen37 enabled auto-merge (squash) August 4, 2026 06:57
@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f1425dec-cb6e-42c9-aeb9-6d7017b6c0ba

📥 Commits

Reviewing files that changed from the base of the PR and between f8f8755 and 6290a2f.

📒 Files selected for processing (5)
  • plugins/me/README.md
  • plugins/me/skills/write-skill/SKILL.md
  • plugins/me/skills/write-skill/agents/openai.yaml
  • plugins/me/skills/write-skill/references/skillopt.md
  • tests/me/me-specific.bats

📝 Walkthrough

Walkthrough

The write-skill skill now provides general skill-authoring and validation guidance. SkillOpt procedures moved to a dedicated reference. Package metadata and tests were updated to match the new structure.

Changes

write-skill generalization

Layer / File(s) Summary
General skill authoring and validation
plugins/me/README.md, plugins/me/skills/write-skill/SKILL.md, tests/me/me-specific.bats
The skill now covers minimal SKILL.md structure, trigger testing, troubleshooting, placement, and conditional SkillOpt use. The README and tests reflect the updated contents and removed agent metadata.
SkillOpt setup and harvesting
plugins/me/skills/write-skill/references/skillopt.md
The reference documents installation checks, terminology, evidence harvesting, provider behavior, scope rules, supported commands, and multi-worktree task preparation.
SkillOpt safety and adoption
plugins/me/skills/write-skill/references/skillopt.md
The reference documents cost, privacy, state safety, protected regions, rollback, reporting, and explicit adoption requirements.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/improve-verify

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@baleen37
baleen37 merged commit cf715d7 into main Aug 4, 2026
3 of 4 checks passed
@baleen37
baleen37 deleted the feat/improve-verify branch August 4, 2026 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant