Repository navigation
Implementation and test details can end up in specs because they have nowhere else to go #1967
Description
Every behavior has three sides:
- Intended: what the system should do (the spec)
- Implemented: how it does it (the code and design)
- Verified: what proves it works (the tests and evidence)
OpenSpec gives a lasting home only to the first.
What we ran into
We reviewed the specs from one of our first changes, a desktop app foundation. They passed validation, but many requirements weren't really behavior:
| In the spec | The actual behavior | Where the rest went |
|---|---|---|
| Framework security settings, named one by one | "Interface code can't access the filesystem or OS processes" | Design doc |
| "Service code must not import the UI framework" | Not behavior at all; it's a code-structure rule | Lint rule |
| "When the service is deliberately blocked… as observed by an external process" | "When the service stops responding, the window stays usable" | Test tasks |
Rewriting them made the specs cleaner. But the details we moved out now sit in the design doc, tasks and lint config, and nothing links them back to the requirements they support. Once the change is archived, nothing tells you how a requirement like "the service exits if the app is killed" is verified, or whether it is at all.
The gap shows up in two ways.
Symptom 1: specs fill up with implementation and test details
This doesn't always happen, but we've run into it: agents sometimes put framework details, component responsibilities and test procedures into requirements. Better guidance helps, but it isn't the root cause. Specs are the only artifact that outlives a change: design and tasks are archived with it. If a detail matters later, the spec is the only place it can survive.
Symptom 2: nothing links the three sides
Even when details go in the right place, you can't trace from a requirement to the code that implements it or the tests that verify it. So you can't easily see:
- which requirements are verified, and how
- which have no adequate test, for example because the test infrastructure can't check them end to end
- what a change will verify when it's proposed, and what it actually verified once done
Why it's hard
Everything is markdown. Requirements have no stable identity beyond their heading text, and markdown has no natural way to express links that tools can check. Any solution needs some way to identify requirements and scenarios that survives edits, plus a convention (or custom syntax and linting) for referring to them.
Directions worth exploring
- Stable identifiers for requirements and scenarios, so anything can point at them.
- Links from tests to specs: tests reference the scenario they verify, which keeps specs clean and lets tooling report coverage and gaps.
- A lasting home for implementation and verification context, so it doesn't have to live in the spec or disappear at archive.
- A verification plan at proposal time: which requirements will be verified by which tests, and where verification will be manual or missing, so the user can decide how to handle gaps.
What we're asking for
This needs careful, deep thinking before any implementation. The directions above are starting points from one experience, not a plan to build. The right answer may be several smaller fixes rather than one. Please work out what actually fits OpenSpec and its users, and treat these suggestions as input, not a spec.
Open questions
- What should the unit of linking be: requirement, scenario, or both?
- Should links live in the specs, in the tests, or in a separate index?
- How much structure can we add to markdown before it stops being easy to read and write?
Related: #900 (generating tests from specs), #1652 (archive requiring evidence), #1966 (skill references)
Activity
[-]Connect specs to the code and tests behind them[/-][+]Implementation and test details can end up in specs because they have nowhere else to go[/+]sonqh commentedon Oct 26, 2026
Hi Tabish, thanks for writing this up. We ran into the same wall on three OpenSpec projects, and your three open questions are the ones we had to answer for ourselves. Here is what we tried and what it did. Maybe some of it is useful.
What is the unit of linking? We went with the requirement and the scenario. Each requirement gets an ID like
REQ-UA-004. Each of its scenarios getsSCN-UA-004-1,SCN-UA-004-2, and so on. A small tool hands out the numbers, and a number is never used twice. When a requirement goes away, its ID goes to a retired list so nobody picks it up by accident.Where do the links live? In the header, as the first word. Nothing else in the file changes:
From there, everything points at the ID. Code carries a comment
// REQ-UA-004. A test carries// Covers: REQ-UA-004and names the scenario IDs it checks. A task line intasks.mdcites[REQ-UA-004]and holds the shell command that proves the task is done. OneRefs:line links a requirement to other requirements or decisions.How much structure can markdown take? About this much, and it turned out to be enough. The ID is part of the header text, so the current parser already reads it.
openspec validate --all --strictandopenspec archiverun unchanged on all three projects. The only new file is a small TOML with the capability codes.Three things got better once the IDs were in:
Refs:target that does not exist. It also flags a task whose files changed after its proof command passed.Some numbers, in case they help:
Everything is MIT and public at [sonqh/spec-ctx](https://github.com/sonqh/spec-ctx): the [ID scheme](https://github.com/sonqh/spec-ctx/blob/main/decisions/ADR-002-id-scheme.md), a
spec-driven-idsschema bundle that archives with the current CLI, and the engine (npx -y spec-ctx). I am happy to help with a design for ID matching insideopenspec archive, if that is a direction you want. I will not open a PR unless you ask for one.Two questions back, when you have a moment:
Related: #743 asks for the same thing from the test side.
feat(skills): a guide for writing a requirement (#2139)