docs(skill-evals): add a convention for recording live eval runs

[thisisnic]

Summary

• The eval suites are never run against a live model in CI, so there was no record of which suites had been run on which model, or how they fared. Adds a short Recording runs convention to the harness README: a per-skill Run history table and four rules.
• Fills in the first tables. Eight issue-* suites now have one, with 11 rows at main 399ab1bc: all eight on Opus, and issue-backlog-stats on Sonnet and Fable too. 10 runs per row.
• Four suites passed every run. The other four surfaced failures:
  • issue-backlog-stats, step-4: fixtures expected the bare skill name, the skill says slash-command form. Fixed in fix(issue-backlog-stats): expect marketplace-form actions in step-4 evals #1677.
  • issue-backlog-stats, step-2: Sonnet treats STALE-CANDIDATE as a class, Opus and Fable treat it as a flag. The skill text says both. Tracked in issue-backlog-stats: STALE-CANDIDATE is listed as a triage class but defined as orthogonal #1678.
  • issue-triage and issue-reproducer: a handful of cases fail 1 to 3 times out of 10 on Opus. No issue filed yet; the rows record the counts so the next run can be compared.
• The earlier issue-reassess-stats row records the pre-fix flake that fix(issue-reassess-stats): pin nature counting in step-2 classify eval spec #1615 addressed.

Design questions I'd like feedback on

• Per-skill tables only, no cross-skill summary page. An earlier draft had a hand-maintained docs/eval-run-log.md listing all 81 suites; it was either a duplicate of the per-skill tables or a lossy digest, so it was dropped.
• Refs are SHAs on main only; branch runs aren't logged. Simpler than tracking branch SHAs that rebases rewrite.
• Model names are whatever the CLI calls them, no cross-provider normalisation.
• Rows are append-only. No pruning rules yet; happy to add them if the tables get long.
• Notes are links only (fix PR, issue, or a one-line reason for a grader flake). Anything longer belongs in the linked issue.

Type of change

• Documentation (docs/, README.md, CONTRIBUTING.md)
• Skill change
• Tool / bridge contract
• Python package
• Groovy reference impl
• Cross-cutting
• Project template
• CI / dev loop
• Other:

Test plan

• prek run passes on the changed files
• prek run --all-files passes
• N/A for evals: no skill or fixture text changes. The rows record runs made on main, not on this branch.

RFC-AI-0004 compliance

No principles touched — documentation only.

Linked issues

• Refs fix(issue-reassess-stats): pin nature counting in step-2 classify eval spec #1615 (fix the issue-reassess-stats row points at).
• Refs fix(issue-backlog-stats): expect marketplace-form actions in step-4 evals #1677 (fix the issue-backlog-stats step-4 rows point at).
• Refs issue-backlog-stats: STALE-CANDIDATE is listed as a triage class but defined as orthogonal #1678 (issue the issue-backlog-stats Sonnet step-2 row points at).

---

Generated-by: Claude Code (Fable 5.1)

🤖 Generated with Claude Code

[thisisnic]

@potiuk - I was going to run some more of these but was after a bit of feedback on the design of this before I get into it more. I've kept it intentionally minimal for the moment, and was a bit hesitant to add a summary page as it'd need to keep in sync with the individuals suites, but alternatively, I could ditch the individual ones in favour of an overall summary page - what do you think?

[potiuk]

Yeah overall summary sounds good :)

[thisisnic]

I ran these on some more evals in the "issues" group as well to get a feel for the kinds of things that come up often; this should be ready for review now.