Skip to content

feat(product-tree): judge change blockers natively - #718

Open
simonheimlicher wants to merge 10 commits into
mainfrom
work/change-444
Open

simonheimlicher wants to merge 10 commits into
mainfrom
work/change-444

Conversation

@simonheimlicher

@simonheimlicher simonheimlicher commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • The change audit judges a stored Change's blockers from its native dependencies and never from comment text.
  • The audit-change runner gains read-blockers and walk-blockers, which read one page of 100 blockers per issue and walk at most 10 issues.

Background

The change audit compared the candidate's blocker list with text in comments. The store holds the blockers as native issue dependencies, so the audit now reads them there. A read that fills its bound returns a blocked result naming the bound, so a partial list never passes as complete.

Changes

  • spx/31-outcomeeng.enabler/32-changes.enabler/changes.md adds the blockers-from-native-dependencies audit assertion and a [test] assertion linking the new test.
  • spx/31-outcomeeng.enabler/32-changes.enabler/tests/test_audit_change_blockers.compliance.l1.py and the harness and fixtures under outcomeeng_testing/ cover both operations, including page-bound-reached at 100 blockers and at 10 issues.
  • src/plugins/product-tree/skills/audit-change/SKILL.md steps 4 and 6 use the new operations, and scripts/audit_change_run.py implements them.
  • src/plugins/product-tree/CHANGELOG.md records the change; the dist/ trees are regenerated from src/.

This PR replaces #656, which targeted the retired layout and is closed unmerged once this PR opens.

Test plan

  • just test spx/31-outcomeeng.enabler/32-changes.enabler/tests passes
  • spx validation markdown, just eval-links, just check-skills, and just docs-check report no errors
  • product-tree:spec-auditor approved changes.md; instructions:skill-auditor approved audit-change

Versioning

  • product-tree: PATCH, written before merge

Validation

  • Touched-scope deterministic verification passes
  • /reload-plugins confirms the change loads in a running session

Refs

https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@claude

claude Bot commented Oct 27, 2026

Copy link
Copy Markdown

BLOCKING [consistency]: src/plugins/product-tree/skills/audit-change/scripts/audit_change_run.py:350-363

Reference: change-standards/SKILL.md line 32 states that a criterion whose evidence is store state, such as the blocker graph of star-relationships, is judged from reads that a separate Lifecycle load supplies, and that a Maturity selection never infers that state from the record alone. change-standards/references/lifecycle.md line 32 names the one blocker read that Lifecycle load supplies: a Change blockers are its native dependencies, read as one page of 100 with gh api repos/store/issues/N/dependencies/blocked_by --method GET -F per_page=100.
Evidence: this PR adds read-blockers and walk-blockers entirely inside audit_change_run.py (new _BLOCKERS_QUERY at lines 351-359, and _issue_blockers at lines 1453-1483), using gh api graphql against a blockedBy connection, not the REST dependencies/blocked_by endpoint lifecycle.md names. audit-change SKILL.md step 7 is the only place this audit invokes change-standards with Lifecycle, and that step only requests read-authority; the new step 4 blocker reads never go through that Lifecycle load. The audit now has two disagreeing mechanisms for the same store fact: its own GraphQL read here versus the REST read lifecycle.md prescribes and report-changes/scripts/report_changes.py lines 450-490 already implement, contradicting the documented statement that blocker-graph evidence comes from the Lifecycle load.
Required: route the new blocker evidence through the Lifecycle loads canonical-state read, or update change-standards/SKILL.md line 32 and lifecycle.md line 32 to name read-blockers and walk-blockers as a second, reconciled supplier of that evidence.

DEBT [consistency]: src/plugins/product-tree/skills/audit-change/scripts/audit_change_run.py:1443

Reference: the review standard requires checking that a new derivation of a value an existing function already derives agrees with it on every documented input; change-standards/SKILL.md line 55 calls blocked_by entries canonical blocker Change identities.
Evidence: the new _blocker_entry function always builds owner_name plus a hash sign plus the issue number (audit_change_run.py line 1443), the full owner/repo-hash-N form, for every blocker including one in the same repository as the Change. The existing read_blockers function in report-changes/scripts/report_changes.py lines 478-483 derives the same identity as a bare issue number when the blocker repository equals the request store, and only uses the owner/repo-hash-N form for a cross-repository blocker. If the blocked_by field stores same-repo blockers as bare numbers, matching the report-changes convention, then audit-change comparing against its always-prefixed identity would mismatch every same-repo entry, causing step 6 of audit-change SKILL.md to false-positive on blockers the candidate already lists.
Required: confirm which identity form the blocked_by front-matter field actually stores, and make both derivations produce that same form.

DEBT [consistency]: src/plugins/product-tree/skills/audit-change/SKILL.md:92-93

Reference: the runner_contract table in audit-change/SKILL.md documents, per block-reason row, exactly which operations and command shapes can raise it; the page-bound-reached row was updated by this PR to add read-blockers and walk-blockers with the new bound text.
Evidence: the command-failed row at SKILL.md line 92, left unchanged by this diff, still lists only the git rev-parse command for every operation except tool-version, read-authority, read-published, and read-relations, and the gh command of read-authority, read-published, or read-relations, omitting read-blockers and walk-blockers even though the new _issue_blockers function at audit_change_run.py line 1464 runs gh api graphql through the same _command path and skips _repository_root exactly like the three listed operations. The unreadable-output row at SKILL.md line 93 likewise was not extended with the three new failure shapes that _blocker_entry, _issue_blockers, and _identity_parts raise: a blocker entry missing repository, number, or state; a blockers page missing nodes or pageInfo; and a blocker identity that is not owner-slash-repository-hash-number.
Required: extend the command-failed and unreadable-output row text to name read-blockers and walk-blockers and their specific failure shapes.

@claude

claude Bot commented Oct 27, 2026

Copy link
Copy Markdown

DEBT [consistency]: src/plugins/product-tree/skills/audit-change/SKILL.md:93-94

Reference: the <runner_contract> blocked-result table documents each reason as applying to a specific, enumerated set of operations and causes per row -- demonstrated by this very diff, which correctly updates the page-bound-reached row to add read-blockers and walk-blockers and their new causes.
Evidence: the command-failed row is left unchanged: "git rev-parse --show-toplevel (every operation except tool-version, read-authority, read-published, and read-relations), an spx command, or the gh command of read-authority, read-published, or read-relations exits nonzero or cannot be executed." But the new _issue_blockers helper, shared by read-blockers and walk-blockers, issues its gh api graphql call with cwd=context.cwd, exactly like _read_authority, _read_published, and _read_relations (none of which call _repository_root, the function that runs git rev-parse) -- so the two new operations likewise skip the git-toplevel check and likewise fail command-failed on a nonzero or unexecutable gh call, yet neither is named in the row. The unreadable-output row has the same gap: it lists causes only for read-authority, read-published, and read-relations, omitting the three new raise sites this diff adds in _blocker_entry (a blockers page holding an entry with no repository, issue number, or state), _issue_blockers (a blockers page carrying no nodes array and pageInfo object), and _identity_parts (a store answer naming a blocker that is not owner/repository#number).
Required: add read-blockers and walk-blockers to the command-failed row's git-rev-parse exemption list and its gh-command-failure operation list, and add the three new blocker-specific causes to the unreadable-output row, so the table stays an exhaustive, accurate catalog of blocked-result causes per operation.

A change audit judges the blocker list and graph of a stored Change from
its native dependencies, and the audit runner reads them within stated
bounds. Verification: not-run; the linked test file does not exist yet.

Refs: outcomeeng/changes#444
Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
Compliance evidence for the audit-change runner's native blocker reads:
one page of 100 blockers per issue and at most 10 issues per walk, with
a blocked result naming the bound. Verification: failing; the runner
operations the tests drive are not implemented yet.

Refs: outcomeeng/changes#444
Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
The audit-change runner gains read-blockers and walk-blockers, which read
a stored Change's native blockers one page of 100 per issue and walk at
most 10 issues. The skill judges the blocker list and graph from them
and never from comment text. Verification: not-run.

Refs: outcomeeng/changes#444
Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
The runner now passes the ruff format check that the quality gate runs.
The dist copies follow the source through the build. Verification:
not-run.

Refs: outcomeeng/changes#444
Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
The audit-change runner now reads a Change's native blockers through the
dependencies/blocked_by REST read that the Lifecycle reference names, with the
same bound of one page of 100, so the blocker evidence of a stored Change has
one read path. A page holding 100 entries blocks naming the bound.

The blocked-result table lists read-blockers and walk-blockers in the
path-rejected and command-failed rows and names the blocker failure causes in
the unreadable-output row. The blockers harness, fixtures, and test replay the
REST answer.

Verification: passing (just test on the changes node tests, just check-skills)
Refs: spx/31-outcomeeng.enabler/32-changes.enabler

Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
The audit-change runner docstring records what read-blockers and walk-blockers
were tested with: the four cases of the blockers compliance test, run over
replayed store answers.

Verification: passing (just test on the blockers test file, just check-skills)
Refs: spx/31-outcomeeng.enabler/32-changes.enabler

Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
@simonheimlicher

Copy link
Copy Markdown
Contributor Author

Debt rulings recorded for head 0a6d583.

Ruling position: Plugins Maintainer, relayed to the Author in the change-444 mail thread (messages 22191 and 22223). The Maintainer ruled every debt finding on touched text fixed.

Entry key Ruling Disposition
src/plugins/product-tree/skills/audit-change/SKILL.md, runner_contract blocked-result table, rows command-failed and unreadable-output fix The rows name read-blockers and walk-blockers and their causes (commit 1c26262f792ed6eac687d2aecd3fb917bd6cd9e3).
src/plugins/product-tree/skills/audit-change/SKILL.md, runner_contract blocked-result table, rows path-rejected and page-bound-reached fix The path-rejected exclusion list names the two operations, and page-bound-reached names the full page of 100 blockers (same commit).
src/plugins/product-tree/skills/audit-change/scripts/audit_change_run.py, module docstring, rule script_test_record fix The docstring paragraph after walk-blockers records the four cases of spx/31-outcomeeng.enabler/32-changes.enabler/tests/test_audit_change_blockers.compliance.l1.py and leaves out the cases with no test (commit 0a6d5831c568aa9cc7ca594efd36b4113f1e8c61).

Blocker identity form: no change. change-standards defines blocked_by as canonical Change identities, and lifecycle.md gives owner/repo#N as the canonical form, which the runner returns for every blocker.

Comment 6452465882 (blocking, REST read of blockers) is addressed in 1c26262f792ed6eac687d2aecd3fb917bd6cd9e3. The branch was rebased onto origin/main eee70f844e110a4ddc57a8b954994483479b1795; the audit-change skill directory is byte-identical to the head before the rebase.

https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR

@claude

claude Bot commented Oct 27, 2026

Copy link
Copy Markdown

DEBT [consistency]: src/plugins/product-tree/skills/audit-change/SKILL.md:132-133

Reference: SKILL.md line 95 (runner_contract reason table): the page-bound-reached row is the only row for read-blockers/walk-blockers whose "Condition" column adds a bound field ("The result adds bound, naming the page size, the kind of entry, and the page count or issue count."). The command-failed row (line 91 pre-diff context) and unreadable-output row carry no bound field for these two operations.
Evidence: Step 4 states, for these same two operations: "Request read-blockers with issue when the declared Maturity is Proposed or Framed, and walk-blockers with issue when it is Sliced or Executable; a blocked result returns BLOCKED naming the bound." This reads as a blanket claim that every blocked result from read-blockers/walk-blockers names the bound, but the runner itself (scripts/audit_change_run.py, _command() at line ~502 and _blocker_entry/_identity_parts at lines ~1436-1496) raises COMMAND_FAILED and UNREADABLE_OUTPUT for these operations with no ResultField.BOUND in their evidence — only the PAGE_BOUND_REACHED branches in _issue_blockers (line ~1480) and _walk_blockers (line ~1526) add bound. The immediately preceding two "a blocked result returns BLOCKED" sentences in the same step, for read-published and read-relations, carry no such "naming the bound" qualifier, so this is a localized claim specific to the new operations that the reason table and implementation do not support for every block reason.
Required: Qualify the step-4 sentence to scope "naming the bound" to the page-bound-reached case only (e.g. "a blocked result returns BLOCKED, naming the bound when the reason is page-bound-reached"), matching the other blocked-result sentences in the same step and the reason table on line 95.

Step 4 of the audit-change workflow says a blocked blocker read names the
bound. Only the page-bound-reached reason carries a bound, so the sentence
names the bound for that reason alone.

Verification: passing (just build-skills, just check-skills)
Refs: spx/31-outcomeeng.enabler/32-changes.enabler

Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
@claude

claude Bot commented Oct 27, 2026

Copy link
Copy Markdown

DEBT [consistency]: src/plugins/product-tree/skills/audit-change/SKILL.md:94

Reference: in the same runner_contract unreadable-output row, every pre-existing clause is tagged with the exact operation(s) that can raise it -- the connection clause is tagged (read-authority) only, the issue-body clause is tagged (read-published) only, and the relations-page clause is tagged (read-relations) only.
Evidence: the new clause reads: a blockers page is not a JSON array or holds an entry with no repository_url, issue number, and state, or a walk meets a blocker whose identity is not owner/repo#N -- and tags BOTH sub-conditions with (read-blockers, walk-blockers). In audit_change_run.py, _read_blockers calls only _issue_identity and _issue_blockers/_blocker_entry; it never calls _identity_parts, the only function whose Blocked call raises the owner/repository#number message. _identity_parts is called exclusively inside _walk_blockers, once for the start issue and once per subsequent pending blocker it meets. So read-blockers can never produce the walk-meets-a-blocker-with-a-bad-identity failure; tagging it to read-blockers overclaims relative to the implementation and departs from this rows own per-clause tagging convention.
Required: split the new clause so the JSON-array / malformed-entry condition stays tagged (read-blockers, walk-blockers), and the bad-identity condition is tagged (walk-blockers) only.

The unreadable-output row of the audit-change runner contract tags the
bad blocker identity clause with read-blockers and walk-blockers. Only
walk-blockers reads a blocker identity, so the clause names that
operation alone. The other clauses of the reason table match the runner.

Verification: passing (just build-skills, just check-skills, just docs-check)
Refs: spx/31-outcomeeng.enabler/32-changes.enabler

Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
@claude

claude Bot commented Oct 27, 2026

Copy link
Copy Markdown

DEBT [evidence]: spx/31-outcomeeng.enabler/32-changes.enabler/tests/test_audit_change_blockers.compliance.l1.py:67

Reference: changes.md states the change audit at Sliced and Executable walks each blockers own native blockers to find a blocker cycle ([audit:blockers-from-native-dependencies]), and the runner own _walk_blockers docstring in audit_change_run.py line 1512 states that issues are read in breadth-first order, each once, so a cycle ends the walk.
Evidence: The only two walk fixtures exercised by the new test file are strictly linear, non-repeating chains: walk-within-bound.json walks issue 12 to 13 to 14 with every blocker a new issue, and walk-past-bound.json walks a 12-issue linear chain 12 through 23, also with no issue revisited. Neither fixture presents a blocker edge back to an issue already in seen, so the dedup branch in _walk_blockers that the cycle-detection assertion depends on to let the walk terminate on a cycle and still return a correct graph is never exercised by any test.
Required: Add a fixture and test where a walked issues blockers loop back to an issue already visited, for example issue 12 blocked by 13 and 13 blocked by 12, asserting the returned graph still terminates within the bound and both entries blockers correctly show the back-edge, so the data the Sliced and Executable cycle-detection step in SKILL.md relies on is verified rather than assumed correct by inspection alone.

The unreadable-output row named only spx verification run for its
non-JSON-line condition, though the GraphQL gh queries of read-authority,
read-published, and read-relations parse through the same line reader.
The path-rejected row omitted count-limits, which reads the candidate
through the same path checks as read-candidate, start, and reconcile.

Both rows now match the runner. The settled consistency entry for the
path-rejected row leaves the node's ISSUES.md.

Verification: passing (just build-skills, just check-skills, just docs-check)
Refs: spx/31-outcomeeng.enabler/32-changes.enabler

Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
The walk fixtures were linear chains, so no case reached the branch of the
walk that skips an issue it has already seen. A new fixture lets two Changes
block each other, and a case asserts that the walk reads each issue once and
ends.

Verification: passing (just test on spx/31-outcomeeng.enabler/32-changes.enabler/tests)
Refs: spx/31-outcomeeng.enabler/32-changes.enabler

Claude-Session: https://claude.ai/code/session_01JsNx5Xn213zFDgN6gbJNdR
@claude

claude Bot commented Oct 27, 2026

Copy link
Copy Markdown

BLOCKING [consistency]: src/plugins/product-tree/skills/audit-change/scripts/audit_change_run.py:1465

Reference: the module docstring states "A filesystem, encoding, or decoding error ends the request with a declared BlockReason, never with a traceback" (audit_change_run.py:18). Every other JSON-decode site in this file enforces that guarantee by catching RecursionError alongside the decode error — see _json_lines (lines 565-590, used by read-authority/read-relations/read-published), _payload_text (lines 724-727), and the top-level request parser (lines 1643-1654, 1723).
Evidence: the new _issue_blockers helper parses the REST gh api .../dependencies/blocked_by response with entries: object = json.loads(stdout) guarded only by except ValueError as exc: (lines 1463-1468). json.JSONDecodeError is a ValueError subclass, so a plain malformed-JSON response is still caught, but RecursionError is not — it is a RuntimeError subclass. A deeply nested JSON body from the store (the same failure mode every sibling parser in this file explicitly guards against) propagates out of _issue_blockers as an uncaught RecursionError, producing a Python traceback on stdout/stderr instead of the declared unreadable-output result the rest of the runner contract promises for read-blockers and walk-blockers.
Required: wrap the json.loads(stdout) call in _issue_blockers to also catch RecursionError and raise Blocked(BlockReason.UNREADABLE_OUTPUT, ...), matching the handling already used by _json_lines, _payload_text, and the request parser in this same file.

@sonarqubecloud

Copy link
Copy Markdown

@claude

claude Bot commented Oct 27, 2026

Copy link
Copy Markdown

DEBT [consistency]: src/plugins/product-tree/skills/audit-change/SKILL.md:158

Reference: src/plugins/product-tree/skills/audit-change/SKILL.md line 90 states "Every blocker finding comes from the read-blockers or walk-blockers result and never from a comment" — the skills own established convention names the exact result field each judgment draws from (e.g. "Every length finding comes from the runner's count-limits result, and every Parent finding from its read-relations result").
Evidence: step 6 says "Judge blocked_by against the blockers rule from the native list alone" for both branches step 4 sets up: at Proposed/Framed the request is read-blockers, whose result carries a flat blockers array (matching the phrase); at Sliced/Executable the request is walk-blockers instead, whose result carries graph, one {issue, blockers} entry per walked issue, with no top-level blockers field at all. Step 6 never states that the candidates own direct-blocker list at Sliced/Executable is the graph entry whose issue equals the candidate (as opposed to, say, the union of every entrys blockers across the walk, which would wrongly include each blockers own transitive blockers). An agent executing this skill at Sliced/Executable has to infer the extraction rule that the read-blockers branch does not need.
Required: name the walk-blockers extraction explicitly in step 6, e.g. "the native list is blockers from read-blockers, or at Sliced and Executable the blockers of the graph entry whose issue is the candidates own issue."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant