Skip to content

spec(audit): require command evidence in a blocked diagnostic - #682

Open
simonheimlicher wants to merge 4 commits into
mainfrom
work/change-101
Open

simonheimlicher wants to merge 4 commits into
mainfrom
work/change-101

Conversation

@simonheimlicher

@simonheimlicher simonheimlicher commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • A started implementation-audit run returns BLOCKED only for a failed command with evidence.
  • Unfinished reading is recovered in bounded ranges and never reaches the blocked diagnostic.

Background

An implementation auditor sealed unfinished reading as BLOCKED, and its dispatcher repaired the result. The blocked-diagnostic assertion now states what a started run's diagnostic carries, so unfinished reading never passes as repairable.

Changes

  • spx/21-spec-tree.enabler/68-audit.enabler/audit.md: the blocked-diagnostic ALWAYS assertion names the failed command, or the absent prerequisite a failed command established, with that command's evidence in the exitCode and stderr lines, in one of three shapes. A started run returns no blocked diagnostic without one of them.
  • spx/21-spec-tree.enabler/68-audit.enabler/audit.md: a new NEVER assertion bars rendering the run driver's own partial, truncated, or unfinished reading as the failed command of a blocked diagnostic.
  • The skill src/plugins/spec-tree/skills/audit-implementation/SKILL.md stays unchanged. Its reconciliation, the two ISSUES.md entries in the node, and the assertion auditor's debt finding on that skill settle with outcomeeng/changes#574.

Test plan

  • spx validation markdown passes
  • just eval-links passes
  • spx spec status --format json exits zero
  • The branch changes only spx/21-spec-tree.enabler/68-audit.enabler/audit.md against origin/main

Refs

  • spx/21-spec-tree.enabler/68-audit.enabler
  • outcomeeng/changes#101

Validation

  • Touched-scope deterministic verification passes
  • Plugin reload is not applicable: the changeset ships no plugin content

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@claude

claude Bot commented Oct 17, 2026

Copy link
Copy Markdown

DEBT [consistency]: spx/21-spec-tree.enabler/68-audit.enabler/audit.md:25

Reference: the amended ALWAYS assertion requires a run that reaches none of the final statuses to return the blocked diagnostic naming the failed command, or the absent prerequisite a failed command established, with that command evidence in the diagnostic exitCode and stderr lines -- a started run returns no blocked diagnostic without one of those three shapes. The new NEVER assertion on line 27 bars rendering the run driver own partial, truncated, or unfinished reading as the failed command of a blocked diagnostic.
Evidence: the governed implementation, src/plugins/spec-tree/skills/audit-implementation/SKILL.md lines 446-450, still reads: A required unit that cannot reach a final status is not a command rejection, so its diagnostic carries the started runToken, the absent prerequisite or failed operation in command, and payloadSource none, payloadKey none, exitCode none, stderr none. That branch is exactly the shape the newly tightened spec now forbids -- a blocked diagnostic issued for a non-final unit with no command exitCode or stderr evidence of any of the three admitted shapes. The PR body confirms the skill stays unchanged and defers reconciliation and the two ISSUES.md entries in the node to outcomeeng/changes#574, but spx/21-spec-tree.enabler/68-audit.enabler/ISSUES.md currently carries no entry naming this specific disagreement -- the closest existing entry, the verdict format lists three BLOCKED causes while stage 7 routes a fourth note at ISSUES.md:394, describes a different gap, missing stage-7 routing, not the exitCode/stderr evidence requirement this change just added.
Required: add an ISSUES.md entry on this node, or land it before or with this spec change, naming the vertical disagreement between the amended compliance assertions and the SKILL.md exitCode none / stderr none BLOCKED branch, so the gap is tracked rather than merged silently.

@claude

claude Bot commented Oct 17, 2026

Copy link
Copy Markdown

BLOCKING [consistency]: spx/21-spec-tree.enabler/68-audit.enabler/audit.md:25

Reference: the amended ALWAYS assertion at line 25: "requires a run that reaches none of those to return the blocked diagnostic naming the failed command, or the absent prerequisite a failed command established, with that commands evidence in the diagnostics exitCode and stderr lines: ... a started run returns no blocked diagnostic without one of those three shapes."
Evidence: src/plugins/spec-tree/skills/audit-implementation/SKILL.md lines 446-450 (unchanged by this PR) state the opposite for exactly this cause: "A required unit that cannot reach a final status is not a command rejection, so its diagnostic carries the started runToken, the absent prerequisite or failed operation in command ... and payloadSource: none, payloadKey: none, exitCode: none, stderr: none." That is a blocked diagnostic with no command evidence at all, which the new ALWAYS assertion explicitly forbids. SKILL.md line 495 still carries the exact pre-amendment phrase this PR replaced in the spec ("naming a concrete failed operation or absent prerequisite"), confirming the skill was not updated to match. The PR body states the skill "stays unchanged" and that reconciliation settles with outcomeeng/changes#574, naming the two ISSUES.md entries in the node as the tracking record - but spx/21-spec-tree.enabler/68-audit.enabler/ISSUES.md contains no entry describing this exitCode/stderr gap or the "unfinished reading sealed as BLOCKED" incident the PR background describes (checked by grep for "exitCode: none", "unfinished reading", "repaired", "574" - no matches). So the spec now declares a contract the governed skill actively violates for a named, reachable cause, with no verifiable in-repo record tying that violation to a tracked settlement.
Required: either update SKILLs md verdict_format (lines 446-450, 495) in this same change to carry real command evidence for the required-unit-nonfinal cause, or add the actual ISSUES.md entry in this node documenting the gap and pointing to outcomeeng/changes#574 so the deferral is verifiable from the repository rather than only asserted in the PR description.

…nostic

State the three command-evidence shapes a started implementation-audit
run's blocked diagnostic carries, and bar an unfinished reading from
standing as the failed command. Remove the two coordination-note entries
this settles.

Refs: spx/21-spec-tree.enabler/68-audit.enabler
Verification: not-run
The skill the notes describe is unchanged by this changeset, so both
entries stay in the node until the skill reconciliation settles them.

Refs: spx/21-spec-tree.enabler/68-audit.enabler
The blocked-diagnostic assertion forbids the skill's no-evidence branch
for an unfinished unit; the node now records the gap and its settlement
condition.

Refs: spx/21-spec-tree.enabler/68-audit.enabler
@claude

claude Bot commented Oct 18, 2026

Copy link
Copy Markdown

DEBT [consistency]: spx/21-spec-tree.enabler/68-audit.enabler/audit.md:25

Reference: the same ALWAYS assertion states the blocked diagnostic carries "that commands evidence in the diagnostics exitCode and stderr lines" and then enumerates exactly three admissible shapes, the third being "a harness-tool failure with the tools name and its error text".
Evidence: shapes 1 and 2 both have a natural value for the exitCode line (a numeric exit code, or a named termination/timeout-bound/signal-number), but a harness-tool failure (eg a Read or file-write tool erroring) is not a subprocess and carries no exit code at all. The assertion names only "the tools name and its error text" for this shape and does not say which of the two evidence pieces goes in exitCode versus stderr, nor whether exitCode: none is permitted the way the pre-existing "required unit" cause explicitly states exitCode: none, stderr: none at lines 446-450 of src/plugins/spec-tree/skills/audit-implementation/SKILL.md. A run driver trying to satisfy this ALWAYS assertion for a harness-tool failure has no documented value for the exitCode line, reproducing the same class of undefined-field gap ISSUES.md already tracks for the reconciler-drift BLOCKED cause ("The verdict format lists three BLOCKED causes while stage 7 routes a fourth").
Required: state explicitly what the exitCode line carries for a harness-tool failure (eg exitCode: none, or the tools exit status where one exists), so the three shapes the assertion names are each fully specified.

The third evidence shape of the blocked diagnostic now gives both
diagnostic lines a value: exitCode none and the tool's name and error
text in stderr.

Refs: spx/21-spec-tree.enabler/68-audit.enabler
@sonarqubecloud

Copy link
Copy Markdown

@claude

claude Bot commented Oct 18, 2026

Copy link
Copy Markdown

BLOCKING [consistency]: spx/21-spec-tree.enabler/68-audit.enabler/audit.md:25

Reference: src/plugins/spec-tree/skills/understand/SKILL.md (<touched_file_debt>): "ALWAYS: fix debt that the current change causes, surfaces, or invalidates. A change invalidates another file when it removes a symbol that file references, enforces a rule it violates, falsifies its guidance, or causes a gate, audit, or review to expose its imperfection. Location never licenses deferral. Record and proceed only for work independent of the current change in a surface the change neither touches nor invalidates, at the correct tier: decision or spec for durable truth, methodology for reusable workflow, a Change for pending work, ISSUES.md for a known defect with its settlement condition."
Evidence: The tightened ALWAYS assertion this PR lands at audit.md:25 requires a started runs blocked diagnostic to carry a failed commands evidence in exitCode/stderr. The PRs own new ISSUES.md entry ("The blocked-diagnostic assertion forbids the skills no-evidence branch for an unfinished unit") states that src/plugins/spec-tree/skills/audit-implementation/SKILL.md lines 446-450 and 495 give exactly the no-evidence exitCode: none, stderr: none branch the new assertion forbids, and concludes "A run driver that follows the skill returns a started-run BLOCKED the assertion forbids." The PR description confirms the skill is left unchanged and defers reconciliation to outcomeeng/changes#574. This is the changes own admission that it enforces a rule SKILL.md violates - the exact invalidation trigger the cited rule names - yet SKILL.md is not fixed in this PR. Deterministically, from the moment this merges, any audit-implementation run that cannot resolve a required unit will emit a blocked diagnostic that violates the newly-merged spec it is supposed to comply with.
Required: Either update the blocked branch of src/plugins/spec-tree/skills/audit-implementation/SKILL.md (lines 446-450, 495) in this PR so it carries one of the three evidence shapes the new assertion requires, or hold the tightened assertion until a companion change makes the skill comply - per the cited rule, recording the gap in ISSUES.md with a settlement condition in a separate, not-yet-merged Change does not license deferring a fix for a file this change itself invalidates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant