Repository navigation
feat: keep agents on spec by wiring in the project's own checks #1983
Description
OpenSpec's value depends on the agent actually following the specs while it writes code. Today that's mostly left to the agent's judgment, and it often drifts. #901 is one example: agents skip reading the existing specs before proposing a change.
Agents already check their own work as they go: they run tests, linters and builds. OpenSpec doesn't use any of that. The apply instructions never tell the agent to run the project's checks, and config.yaml has no place to say what those checks are. So "done" means "tasks ticked," not "the code passes and matches the spec."
Ideas
| Idea | What it does |
|---|---|
Project checks in config.yaml |
Declare the project's test, lint and build commands once, e.g. checks: { test: "make test", lint: "make lint" } |
| Apply runs them | Apply tells the agent to run the declared checks after each task and fix failures before ticking it off |
| Read the specs first | Apply and propose name the exact main specs the change touches and tell the agent to read them before writing anything (#901) |
| Trace scenarios to tests | Every scenario the agent implements should have a test, reported by verify (builds on #1979) |
Setup check in openspec doctor |
Warn when the project has no declared checks, no tests, or an agent instructions file too big to be useful |
Out of scope
- OpenSpec running the checks itself or blocking archive on them. This issue is about steering the agent, not replacing CI.
- Judging whether the code is good. Only whether the agent did what the project and spec ask.
Open questions
- Should checks be advisory guidance only (like
operations.apply.guidance), or also somethingopenspec verifyreports on? - Can we detect likely commands (
package.jsonscripts,Makefiletargets) duringinitand offer to fillchecksin?
Reacted by Pierre Boissinot
Activity
clay-good commentedon Oct 1, 2026
A user building agent skills (not code) with OpenSpec asked whether verify can test their Given/When/Then scenarios against the generated
SKILL.md.The design ask: extend "trace scenarios to tests" beyond code to any deliverable that has its own way of being checked. Code is checked by unit tests and infrastructure as code (Terraform) by
planor policy checks. An agent skill is checked by evals, which run each scenario as a prompt against the skill and report pass/fail. A declared check should be able to be any of these, so verify can confirm each scenario was actually exercised, whatever the deliverable is.Docs side of the same request: #2023.
feat(skills): lessons from open issues and PRs: check the plan, name …