Skip to content

ci(bench)!: take the bench off the pull-request path - #316

Merged
ryanleecode merged 55 commits into
mainfrom
stryker/bench-off-pr
Oct 13, 2026
Merged

ci(bench)!: take the bench off the pull-request path#316
ryanleecode merged 55 commits into
mainfrom
stryker/bench-off-pr

Conversation

@systemfsoftware-maker

@systemfsoftware-maker systemfsoftware-maker commented Oct 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

No pull request runs the bench any more. bench.yml runs on each push to main and on workflow_dispatch. A plan job picks side A and the legs: on a push, side A is the lane's last green main commit, and a leg runs only when Turbo reports its package, or an engine directory every leg builds, affected since then. The full corpus runs on a dispatch, when no green ancestor exists (side A is then the first parent), and when the lane's own inputs changed. Time is reported, never enforced: GitHub's job timeout is the only kill, 20 minutes on each leg job and 60 on the plan job. When a leg's total regresses, the verdict names the commit, its base, the metrics and root's next action.

This is layer 1 of the split in docs/plans/2026-10-13-0954-ci-bench-off-pr-plan.md, which supersedes the 0749, 0711, 0658, 0629, 0554 and 0435 plans and, through them, the 1800, 1854, 1926, 2032, 2154, 0223, 0238, 0258, 0416 and 0434 plans. Layer 2 (the enterprise install) is judged separately and stacks on this one. #312 stays open and untouched. Most hunks here are ported from it, without its run-identity gate, provisional reports and lockfile pins.

Breaking: bench-report.json moves from 1.1 to 2.0. The setup-timed-out and budget-exceeded abort codes and the timed-out run exit are gone, and the report adds ref, leg and the regression block.

CONST-W3 declaration: AGENTS.md:7 lists .github/workflows/ as Read-only, and this PR edits .github/workflows/bench.yml. The edits are:

  • the trigger: push to main plus workflow_dispatch, with pull_request and paths-ignore removed, and both checkouts at github.sha;
  • concurrency and job caps: bench-${{ github.event_name == 'push' && github.sha || github.run_id }}, no cancel, timeout-minutes: 20 on the leg job and 60 on the plan job;
  • the plan job: a full-history checkout, the harness install and pnpm --filter @systemfsoftware/stryker-bench plan, which outputs side A and the legs; actions: read, so it can list the lane's own runs; and the bench job's if, which skips it when no leg is selected;
  • the sides: worktrees bench-a at the plan job's base and bench-b at HEAD, each with its own released tarballs and install;
  • the checkout: harness install only, and the Turbo cache keyed on github.sha and passed as BENCH_TURBO_CACHE_DIR;
  • the report steps: JOB_STARTED_AT, BENCH_REF, and uploads on always();
  • the header: the lane's measured runner-minutes, run shape and PR-CI effect.

The grant is Ryan's (root) intent of 2026-10-12 to take the bench off the PR path. CONST-E9 does not apply: Bench grades main's commits, not this PR. No other workflow file is touched.

What changes

  • Trigger: push to main plus workflow_dispatch, with no path filter. Concurrency is bench-${{ github.event_name == 'push' && github.sha || github.run_id }} with cancel-in-progress: false, so a push groups by its commit and a dispatch by its run id. Each leg job has timeout-minutes: 20 and the plan job 60 (root's E10 ruling, cf39161d). The workflow opens with its cost header (see Bench cost).
  • Plan job: two pure decisions choose the comparison. chooseBase (test/bench/src/choose-base.workflow.ts) picks side A: on a push, the newest green Bench push run on main (by run creation time) whose commit is an ancestor of github.sha and is not github.sha, from the gh api listing and git merge-base --is-ancestor; with none, or on a dispatch, the first parent. selectLegs (test/bench/src/select-legs.workflow.ts) then picks the legs from the diff against side A and turbo query affected --packages. The lane's own inputs are test/bench (as Turbo reports it), bench.yml, .github/actions/released-tarballs/ and the flake pin; a change to any of them runs the full corpus. The job writes its choice and its wall time to the step summary. It fails when the checkout has uncommitted changes, because Turbo adds them to the affected set, and when a commit has neither a green ancestor nor a first parent. When no leg is selected, the bench job is skipped and the run is green.
  • Sides: side A is the plan job's base and side B is github.sha. Each is a worktree under RUNNER_TEMP with its own released tarballs and its own pnpm install --frozen-lockfile, and each is timed. The checkout holds only the harness.
  • No deadlines: BENCH_DEADLINE_MS, the setup budget, the run deadline and BENCH_RUN_TIMEOUT_MS are all deleted, along with the report shapes only they produced.
  • Wall time: the job's first step records JOB_STARTED_AT. The report and step summary show the leg's wall time against a 10-minute target (LegTargetMs). Going over changes neither the exit code nor the verdict level.
  • Verdict: only a leg's total decides a regression. It needs non-overlapping four-run samples and |Δ median| >= max(3% of A's median, 10 ms). The regression block lists total first, then every other phase of that leg that regressed. On refs/heads/main the next action is "root opens a revert PR of the culprit or a root-fix P0"; on any other ref it is "dispatched on : speed evidence only, no revert". The report's regression is a tagged union, PhaseRegressed or NoLegRegressed. A regression is summarized and exits 0 with an ::error line. Only failed/aborted exit 1.
  • Fixed run path: every run executes from RUNNER_TEMP/bench-run. The side's tree is renamed there for the run and back afterwards, including when the run fails. The leg aborts with run-path-occupied, run-path-move-failed or run-path-unrestored. Bench configs now reference plugins relative to the config file, so they still resolve after the move. On the old two-root layout, A/A run 38135504932 recorded different ids per side and reported a false regression of +1311.5 ms (3.5%) on the typescript-checker leg.

Turbo compliance

This PR adds no hash cache, change classifier or cache gate, and it removes one: the hand-written paths-ignore (**/*.md, docs/**) on the bench trigger. Evidence is from turbo query (turbo 2.10.12) on this branch:

  • paths-ignore versus Turbo. Over 11 sampled main commits, paths-ignore skips none. turbo query affected --packages <leg packages> --base <parent> --head <sha> --exit-code skips every leg on 61d40db4, ee2c4cfb and 3495e21d, and runs only the affected legs on ddeb67ab (e2e-core and enterprise) and 3cb13423 (enterprise). This PR uses that only as evidence: every push to main benches every leg, because the acceptance is one comparison per main SHA. Skipping legs Turbo calls unaffected is deferred to root (plan KD11).
  • Builds are Turbo's. Each side builds through turbo run build with one shared --cache-dir. The enterprise closure comes from turbo run build --dry=json. Task hashes for @systemfsoftware/stryker-js's closure (15 tasks) are identical in the checkout and in a worktree at another path, so the shared cache hits by content only.

Remaining bespoke mechanisms, each with its rule-4 reason:

  • .sfs-deps released tarballs, cached on the stryker-published flake pin (the released-tarballs action and the same-pin step). These are Nix store paths: a Nix flake output that Turbo neither builds nor hashes. A pin change that matters also changes pnpm-lock.yaml, which Turbo reports as LockfileChanged (6c6eaaa9).
  • actions/cache transport of .turbo/cache. The content is Turbo's own task cache. The repo has no Turbo remote cache: no TURBO_TOKEN, TURBO_TEAM or TURBO_API in any workflow, and no remoteCache in turbo.json. Enabling one needs a credential and belongs to Stream K's repo-wide audit, which also covers the same transport in ci.yml, mutation.yml and kill-matrix.yml.

Not caches: workload-digest.service.ts only proves both sides ran one workload and skips no work, and the pristine-tree restore resets the tree between runs. The bench runs the CLI with --full (run-bench.service.ts:90), so Stryker's incremental reuse plays no part in a measured run.

Tests and root's testing law

Root's law of 2026-10-12: every new or changed integration test drives the real product through its public surface on a real project and asserts only observable outcomes, and a pure decision gets property laws beside it.

Integration tests: none are new and none remain changed. This PR had to change one line of test/e2e-core/tests/bench-decisions.integration.test.ts (ReadBenchRunCommand's exit field). That file did not drive the product. It fed hand-written NDJSON lines, corpus text and report text to the pure readBenchRun, coveringTestFiles and BenchCorpusJson, and asserted decision fields. Under the law, and CONST-T12/T14, it is a decision test filed as an integration test, so 0db0ffa8 deletes it. A compliant replacement would run the bench CLI on a real corpus. That is the Bench lane itself, which runs on main and on dispatch, and its evidence is the A/A run below.

Pure-core property tests, each beside its workflow and drawing production schemas:

  • test/bench/src/__tests__/, new, with the modules moved there under the scope law (below):
    • measure-leg;
    • plan-run-path, including a cwd equal to the tree, the enterprise shape;
    • settle-run-path.
  • test/e2e-core/src/__tests__/, changed:
    • summarize-bench: total decides, the 10 ms floor, and a phase-only regression gives no block;
    • render-bench-report: next action by ref, exit and level independent of wall time, and the phase-only slowdown notice;
    • read-bench-run: the timed-out laws are removed with the shape. Laws re-homed from the deleted file cover the cost sum over drawn MutantCost, the 1-based number of the undecodable line among blank and decodable lines, a verdict count other than one, and phase durations carried or refused;
    • covering-test-files, re-homed: sorted distinct holders, and non-JSON text refused.

test/bench test lane. It now has a test script, a vitest.config.ts with inSourceSchemaLaws() in the same shape as test/checker-parity, and tsconfig.test.json. Root's check runs it through Turbo with every other package's test. It is not in the mutation lane: mutation.yml:36 hard-codes PROJECTS, which Stream K owns, so the moved workflows have laws but no mutation grading until K enrols test/bench.

Not re-pinned, because another suite or the type already holds them:

  • the legacy-verdict not-recorded decode, pinned by packages/stryker-js-cli-contract/tests/run-event-wire-line.integration.test.ts:324-338 (the same RunEventWireLine codec readBenchRun uses);
  • the corpus empty-repo refusal: bench-corpus.schema.ts moved into test/bench/src, its only consumer (8689134a), and its in-source block checks that every text field refuses only the empty string and every list, the repo list included, refuses only the empty list.

Placement under root's scope law (2026-10-13)

The law: no new code goes into test/e2e-core, no commit or PR scope is named e2e-core, and no new core, common or shared test-tooling package is created. Stream D owns dissolving test/e2e-core.

  • Moved to test/bench/src (e28351fc). run-path.schema.ts, plan-run-path.workflow.ts, settle-run-path.workflow.ts and measure-leg.workflow.ts, with their laws. test/bench is their only consumer, and they import BenchSide and LegTime from @systemfsoftware/stryker-e2e-core. test/e2e-core/src/mod.ts is back to main.

  • New leg-clock.schema.ts. measureLeg now decides a bench-local LegClock (LegElapsed or LegClockBehind), and main.ts maps it onto the report's LegTime. The reason: make-body-purity forbids a decision from constructing a class imported from another package, and e2e-core cannot import test/bench.

  • Left in e2e-core for Stream D. Edits to six pre-existing modules stay where they are:

    • additions that a pre-existing declaration in the same file consumes, in bench-report.schema.ts, bench-summary.schema.ts and render-bench-report.workflow.ts;
    • deletions of the timeout failure code and RunExit, in bench-run.schema.ts and read-bench-run.workflow.ts;
    • the 10 ms floor in summarize-bench.workflow.ts, which rescaled the pre-existing ∀abk law.

    Their four law files stay beside their modules, and the re-homed laws must not be lost (root, 2026-10-12).

  • Scope names. The PR title is ci(bench)!. Commit 0db0ffa8 is titled test(e2e-core), and renaming it needs a force push, which this branch never does. Main squash-merges under the PR title, so that subject never reaches main.

  • Counter-critic. One adversarial-reviewer round (served by minimax-code/MiniMax-M3.1-Flash-Preview, with no fallback) upheld every placement decision. It corrected the e2e-core rationale above into that three-way split, which is now in the plan.

  • Result for the 19 files (root's merge-cycle check, 2026-10-13).

    • At e5844df9 the PR changed 19 files under test/e2e-core: 7 added, 11 modified and 1 deleted. e28351fc moved all 7 added files to test/bench/src, so at a359a82d the PR adds no file under test/e2e-core (git diff --name-status 25120175 a359a82d -- test/e2e-core: 10 M, 1 D).

    • The 10 modified files can't move either: every export this PR adds to them is used by a declaration that already exists on main, inside e2e-core. Moving any of them would mean moving those modules, which is Stream D's work.

      New export Its e2e-core consumer that already exists on main (25120175)
      LegTargetMs, LegTime (an S.TaggedUnion of LegMeasured and LegNotMeasured) BenchReport (bench-report.schema.ts:50, now leg: LegTime), renderBenchReport (render-bench-report.workflow.ts:268, legText at :274)
      RegressedMetric, PhaseRegression BenchSummary (bench-summary.schema.ts:100, regression at :120), BenchReportOutcome (bench-report.schema.ts:69, summarized.regression at :75), summarizeBench (summarize-bench.workflow.ts:391), renderBenchReport
      BenchRunFailureCode (narrowed in place) BenchRunInvalid.code (bench-run.schema.ts:52, now :48)

    • Every other line this PR adds to those files is either a non-exported helper inside a workflow that already existed, or one in-source refusal block per refined file (next section). No code needs to move.

Pack compliance round (root, 2026-10-13)

Root required every confirmed pack violation in the a359a82d full-path receipt to be fixed. None was merged as is or waived. Each fix is its own pushed commit:

# Rule Fixing commit What changed
8 cell-architecture four-channel-contracts 26c352b9 RunPathUnrestored carries cause: S.optional(S.Unknown) and the run's end. It no longer holds Cause.pretty strings. The service passes each Exit's Cause, and the text is rendered once, in the error's message, at the report edge.
11 schema-laws tagged-unions-over-state-by-presence 26c352b9 The settle command takes RunPathMoveBack (MovedBack or MoveBackFailed { cause }) and RunPathRunEnd (RunSucceeded or RunFailed { cause }), and matches on the move-back. No nullable state fields remain.
9 schema-laws data-only-schema-classes 08eff64a LegTime is an S.TaggedUnion. Two classes and a type id are gone, and the encoded shape is unchanged.
10 schema-laws invariants-as-refinements f501c320 LegElapsed.wallMs is finite and >= 0. BenchReport.ref is S.NonEmptyString, and BENCH_REF is read with Config.NonEmptyString, so an empty value aborts as environment-incomplete.
12 schema-laws refusals-beside-generated-laws 486ae9b4 One in-source refusal block per refined file. Each draws the unrefined base, appends boundary values, compares with a domain predicate, and targets a private predicate. In test/bench that is 5 files; in e2e-core, 4 files (boundary below).
15 ci-wall-clock heavy-suites-judge-every-main-commit 7d962fca A dispatch takes its own concurrency group (github.run_id).
14 ci-wall-clock heavy-lanes-state-their-daily-minutes 913fcbf9 bench.yml opens with the cost header. 5a088c60 re-priced it for the affected-only selection.

#13 (surface-changes-are-versioned) stays rejected: @systemfsoftware/stryker-e2e-core is private.

The e2e-core boundary (plan KD13). Two fixes had to touch pre-existing e2e-core files. #9 did because LegTime is declared there and consumed by BenchReport. #12 did because the rule puts a refusal in the refined schema's own file and bans a separate refusal file. Each took its narrowest form:

No e2e-core file, export, dependency, config or helper is added. The mechanism already exists there: annotation.schema.ts:107, and includeSource at vitest.config.ts:12. Stream D moves the blocks with their files.

Deviations, recorded in the plan:

  • The settle law asserts the carried causes, not the message text. An oracle for the text would copy Cause.pretty.
  • The pack's "widen the refinement once" check was not run, because root forbids hand-applied probes.

Evidence

  • CI on the exact head 227773f4 (contains main cec409b1, ci(ci): apply scripts/deno.json permission grants in the repo scripts tests #338's -P at ci.yml:137; checked with git merge-base --is-ancestor), all green, attempt 1, nothing rerun:

    • CI 38416467229: every job success (cache-sweep skipped), including check (115365565303), checker-parity (1)-(6) ((4) is 115365665126) and checker-parity-report. In check the Gate ran: GATE: success, PLANTED: success, Turbo 18 successful, 18 total.

    • Changeset Check 38416468148, Nix 38416467685 and Commitlint 38416467054: success.

    • The Turbo pin ran and passed in the Repo scripts tests step (check job 115365565303, step Run ./bin/deno test -P --config=scripts/deno.json scripts/):

      2026-10-13T14:57:22.5623601Z running 3 tests from ./scripts/turbo-affected-packages.test.ts
2026-10-13T14:57:22.6627022Z a changed package is reported as its repo-relative directory, with no './' and no trailing '/' ... ok (100ms)
2026-10-13T14:57:22.7559948Z a change to a root file outside every package is reported as the empty path ... ok (93ms)
2026-10-13T14:57:22.8459928Z a commit that changes no file reports no package ... ok (90ms)
2026-10-13T14:57:22.9339702Z ok | 13 passed | 0 failed (534ms)
  • Local, on f13b9920 (base 2083ba43, merged with main 381b3f5b):
    • tsc -b, oxlint and lint:tsgo exit 0 in test/bench, and vitest run passes 18 files and 107 tests;
    • test/e2e-core is unchanged since feb50851, where tsc -b, oxlint and lint:tsgo exit 0 and vitest run passes 44 files and 367 tests;
    • dprint check exits 0.
  • Sabotage checks this round. chooseBase taking the first listed run, and chooseBase not excluding the pushed commit, each fail the push law (shrunk counterexamples). summarizeBench deciding NoLegRegressed when a leg regressed fails five laws across summarize-bench and render-bench-report. Each file was restored and its suite passes.
  • Plan program smoke runs. On this checkout: a reversed real ci.yml green-run listing (side A is the newest ancestor, not the first listed), an empty listing and a dispatch (both take the first parent), a commit whose parent is a lane input (full corpus), and a failing gh (one escaped ::error line carrying the last 4096 characters).
  • Earlier, on b30908ce (base 25120175):
    • tsc -b, oxlint and lint:tsgo exit 0 in test/bench and test/e2e-core;
    • dprint check and gritlint check exit 0;
    • vitest run passes 10 files and 44 tests in test/bench, and 45 files and 375 tests in test/e2e-core. That is the 11 new refusal properties, less 3 generated codec laws for the two deleted LegTime classes.
  • Sabotage check on the new draw. With path === tree dropped from planRunPath, the plan-run-path identity law fails. Restored, it passes.
  • Earlier sabotage check. The phase-only slowdown law fails when the old renderer is restored with git stash.
  • Not run locally: the bench, mutation testing, or the full e2e lane.
  • A/A on head 896a494c: run 38297580066 (dispatched by the sub-conductor), all 5 jobs green.

    • Side A is the merge's first parent 17e21b47. It is product-identical: the merge brought in ci(e2e): run the e2e legs on every main push instead of on pull requests #314, which is workflows and docs only.

    • On every leg the report is schema 2.0 and summarized, with regression: null and workload same.

    • All 8 runs share one project root, under RUNNER_TEMP/bench-run.

    • On every leg both sides recorded the same inputs digest and the same mutant-id digest:

      Leg Inputs digest / mutant-id digest (same on both sides) Mutants
      typescript-checker 266e2253 / 241c14a1 49
      vitest-runner 6129db89 / 60bd6521 58
      e2e-core b518457e / 85be64a8 39
      enterprise 991132ef / 4d840b34 123

    • Every total reads no-signal: +115 ms, −419 ms, −2 ms and −418.5 ms.

    • Leg wall times against the 10-minute target: 7.5, 4.9, 3.6 and 3.6 min.

  • A/A on head 9b518ba7: run 38375500452 (dispatched by root), all 5 jobs green. This closes E8 and E11. It is the executed evidence for atRunPath's renames, and it covers every runtime change since 896a494c.

    • Side A is the merge's first parent f13b9920. It is product-identical: the merge brought in ci(repo): gate change intents on turbo's typecheck task graph #335, which changes release.jsonc only.

    • On every leg the report is schema 2.0 and summarized, with regression: NoLegRegressed and workload same.

    • On each leg all 8 runs share one project root, under RUNNER_TEMP/bench-run.

    • On every leg all 8 runs, both sides, recorded the same inputs digest and the same mutant-id digest (SHA-256 of the sorted mutant ids, one per line):

      Leg Inputs digest / mutant-id digest (same on both sides) Mutants
      typescript-checker 266e2253 / 241c14a1 49
      vitest-runner 6129db89 / 60bd6521 58
      e2e-core b518457e / 85be64a8 39
      enterprise 864392bd / 4d840b34 123

    • Every total reads no-signal: −254 ms, −196.5 ms, +202 ms and +20.5 ms. No other phase separates; e2e-core's check is not-measured.

    • Leg wall times against the 10-minute target: 7.6, 3.0, 1.9 and 2.0 min, all under the new 20-minute leg cap.

    • The mutant-id digests equal run 38297580066's on all four legs, and so do three of the four inputs digests; the enterprise inputs digest differs from that run's 991132ef, while both sides of this run agree on 864392bd.

    • Reports read from the run's bench-report-* and bench-runs-* artifacts.

Review

Merge-bar receipt: correctness lens aabe4c2a..227773f4 (exact head)

Root named this lens as the remaining merge blocker; it supersedes the feb50851 range as the head-covering receipt. One named, report-only correctness lens, run id 20261013-160821-9393a660, reviewed exactly aabe4c2a22a9ea0dcc929457fc4aff728ea7e804..227773f4b31032c38aefe1bdc8f6de085a810146 on a clean checkout of the head. The range holds:

The merge is conflict-free: git show --cc 227773f4 is empty, and the tree equals git merge-tree of both parents. main's side touches no PR-owned file. Its only workflow or script hunk is cec409b1; the test/e2e-core fixture edits are #326's. Artifacts: /tmp/compound-engineering-0/ce-code-review/20261013-160821-9393a660/correctness.json, full.diff (range log, 4646f883, the combined diff, cec409b1 and the overlap facts), files.txt and served-models.json.

Served model: the lens ran on correctness-reviewer-minimax. Its session record shows one model_change to minimax-code/MiniMax-M3.1-Flash-Preview and MiniMax M3.1 for all 9 assistant turns (provider minimax-code), with no error, no 429, no fallback, and no Grok, xai-oauth or Claude call.

Result: 0 findings. It checked every timeout claim against bench.yml (plan job timeout-minutes: 60 at :42, leg job 20 at :82) and test/bench/README.md. It did not execute the bench or CI.

Recorded item Kind Disposition
test/bench/src/plan.ts:115 prints "GitHub's 60-minute job timeout is the only kill" in the plan job's step summary residual risk No change: the sentence describes the plan job, whose timeout is 60 (bench.yml:42), so it is correct. The lens notes only that a reader could mistake it for the leg claim.
No automated check ties the timeout text in the description, README and plan to bench.yml's values testing gap Accepted, no change. A test asserting prose against workflow YAML would be a source-text test, which the test admission gate refuses.

Limitation: the lens did not re-raise the stale KD22 sentence at docs/plans/2026-10-13-0954-ci-bench-off-pr-plan.md:101 ("ci.yml:137 runs ... without -P"). That line is unchanged in this range, but the merged cec409b1 made it false, and the prompt asked about such statements. It stays the open P3 confirmed by lens 20261013-160030-440b1a8e below, with no fixing commit, because the remaining rounds are body-only. E7 and Evidence record the current state.

Pin evidence on the exact head (verified by root and in this session): CI run 38416467229, check job 115365565303, ran ./bin/deno test -P --config=scripts/deno.json scripts/, executed 3 tests from scripts/turbo-affected-packages.test.ts, all ok, and ended ok | 13 passed | 0 failed (534ms).

Earlier delta correctness lens: feb50851..227773f4

This receipt, with the full-path receipt of feb50851 below, covers the exact head 227773f4b31032c38aefe1bdc8f6de085a810146. One named correctness lens, run id 20261013-160030-440b1a8e, reviewed exactly feb50851dfb6417eee0486e3c1f90ed2538e7b5e..227773f4b31032c38aefe1bdc8f6de085a810146 on a clean checkout of that head. The range holds 8 PR commits (a888bb61, 66086e27, 5f437647, f13b9920, 6e7a210c, cf39161d, a0c3d693, 4646f883) and 4 merges of main (e972b7da, 9b518ba7, aabe4c2a, 227773f4) bringing in 11 main commits, which the lens excluded. Its diff is restricted to the 16 files the PR commits touch. Artifacts: /tmp/compound-engineering-0/ce-code-review/20261013-160030-440b1a8e/correctness.json, full.diff, files.txt and served-models.json.

Served model: the lens ran on correctness-reviewer-minimax. Its session record shows one model_change to minimax-code/MiniMax-M3.1-Flash-Preview and MiniMax M3.1 for all 23 assistant turns (provider minimax-code), with no error, no 429, no fallback, and no Grok, xai-oauth or Claude call.

Result: 1 finding, P3; no behavioural defect. It traced all 8 PR commits and found the schema moves keep every call site, and tailOf keeps the same tail semantics for both callers.

# Sev Finding Location Disposition
1 P3 Plan claims CI omits -P; ci.yml already passes it docs/plans/2026-10-13-0954-ci-bench-off-pr-plan.md:101 Confirmed on 227773f4: KD22's second bullet still says ci.yml:137 runs without -P and that root merges once the owner PR lands, which was true when written and is stale since #338 (cec409b1) merged in. No fixing commit: root ruled this round a body-only edit with no push. The current state is recorded in E7 and Evidence (pins ok in run 38416467229). The one-sentence plan fix waits for root's next push authorization.

Residual risks it recorded: the consumers of the 5f437647 and 66086e27 moves were checked by text search only (no symbol-aware references); the 20-minute leg cap rests on measured job-minutes (longest 7.68) with no measured cold-cache push distribution. Testing gap: the pins' red/green status on the merged head was, in the plan, narrative only. Evidence now quotes the CI log line instead.

Limitation: this is one lens, not a 12-lens full-path run. Root asked for exactly one named correctness lens over this range. The pack and persona lenses' last full run is the feb50851 receipt below.

Full-path receipt: head feb50851, range 2083ba43..feb50851, 47 changed files

The review is one ce-code-review run on the full path (mode:agent depth:full base:origin/main), report-only, run id 20261013-071850-b2fe30db. Its artifacts are in /tmp/compound-engineering-0/ce-code-review/20261013-071850-b2fe30db/: review.json, metadata.json, stages.jsonl, the per-lens JSON, the pack slices, validator-verdicts.json and served-models.json. All 12 reviewers and the validator returned, and none timed out. On feb50851, CI (38361719238), Nix (38361718942) and Commitlint (38361718958) passed; Changeset Check (38361719714) failed for a reason outside this PR (E9).

Verdict from ce-code-review: Ready with fixes. The review confirmed one P1, #9, the Turbo --packages pin. Root ruled on it (E7) and it is now fixed in 6e7a210c. Every other confirmed finding has a fixing commit pushed after the review (table below), except #3, which root closed with A/A run 38375500452 (E8).

Served model per lens. Each model is read from that subagent's session record: its model_change events and the model on each assistant message. Every lens ran on a session-local *-minimax copy of its persona (KD19). Each started on MiniMax M3.1, took one Token Plan 429, and continued on DeepSeek V4.1 Flash through the profile's fallback chain; the validator stayed on MiniMax M3.1. No call went to xai-oauth or Grok.

Lens omp agent Served (assistant turns) Final answer from
correctness correctness-reviewer-minimax MiniMax M3.1 (4), DeepSeek V4.1 Flash (111) DeepSeek V4.1 Flash
testing correctness-reviewer-minimax MiniMax M3.1 (9), DeepSeek V4.1 Flash (47) DeepSeek V4.1 Flash
reliability correctness-reviewer-minimax MiniMax M3.1 (7), DeepSeek V4.1 Flash (44) DeepSeek V4.1 Flash
project-standards generalist-reviewer-minimax MiniMax M3.1 (7), DeepSeek V4.1 Flash (32) DeepSeek V4.1 Flash
maintainability generalist-reviewer-minimax MiniMax M3.1 (3), DeepSeek V4.1 Flash (69) DeepSeek V4.1 Flash
adversarial (in-process) adversarial-reviewer-minimax MiniMax M3.1 (10), DeepSeek V4.1 Flash (36) DeepSeek V4.1 Flash
packs: cell-architecture generalist-reviewer-minimax MiniMax M3.1 (7), DeepSeek V4.1 Flash (22) DeepSeek V4.1 Flash
packs: boundary-testing + mutation-discipline generalist-reviewer-minimax MiniMax M3.1 (9), DeepSeek V4.1 Flash (16) DeepSeek V4.1 Flash
packs: schema-laws generalist-reviewer-minimax MiniMax M3.1 (8), DeepSeek V4.1 Flash (17) DeepSeek V4.1 Flash
packs: package-topology generalist-reviewer-minimax MiniMax M3.1 (8), DeepSeek V4.1 Flash (17) DeepSeek V4.1 Flash
packs: ci-wall-clock generalist-reviewer-minimax MiniMax M3.1 (3), DeepSeek V4.1 Flash (19) DeepSeek V4.1 Flash
learnings: docs/solutions generalist-reviewer-minimax MiniMax M3.1 (5), DeepSeek V4.1 Flash (56) DeepSeek V4.1 Flash
validator (#8, #9, #11, #12, #3, #10, #2, #6) correctness-reviewer-minimax MiniMax M3.1 (11) MiniMax M3.1

The merge leaf is not a lens and decides no verdict. It ran on docs-engineer, which the operator profile serves on Claude Haiku 4.5 (43 turns) (E4, still open). The cross-model peer did not run, because its configured route is xai-oauth/grok; the adversarial lens ran in-process instead.

Corrections made by the dispatch context. The merge leaf anchored pack finding #12 at bench-plan.schema.ts:72 (TurboAffectedPaths, which is rooted at S.fromJsonString and has its generated laws). The schema-laws slice quotes :41 (GreenRunsJson), so #12 was restored to :41 before validation, and the validator was told. The merge leaf also selected only the four P1s for validation. Per Stage 5b step 2, the dispatch context re-selected every P0/P1 plus every actionable finding, eight in one batch: #8, #9, #11, #12, #3, #10, #2 and #6. P3 #7 was over the cap and is unvalidated.

Packs. Graded from systemfsoftware/systemfsoftware main 763fd663:

  • declared in .compound-engineering/config.yaml: cell-architecture, boundary-testing and schema-laws;
  • named by root but not declared: package-topology, mutation-discipline and ci-wall-clock.

Every rule file was ruled: 54 rules in all. 32 are matched by the diff (26 comply, 5 violate, 1 pre-existing) and 22 are not matched.

Pack Rule Ruling Evidence (feb50851 file:line) c0cf821b receipt
cell-architecture callable-vs-resource-syntax comply test/bench/src/run-bench.service.ts:133, test/bench/src/run-bench.service.ts:147 -
cell-architecture decode-never-cast comply test/bench/src/plan-legs.ts:60, test/bench/src/plan-legs.ts:106 -
cell-architecture four-channel-contracts comply test/bench/src/run-path.schema.ts:68, test/bench/src/run-path.schema.ts:27 -
cell-architecture handle-state-privacy comply test/bench/src/run-bench.service.ts:85 -
cell-architecture pipeline-composition comply test/bench/src/run-bench.service.ts:288 -
cell-architecture pure-decision-workflows comply test/bench/src/choose-base.workflow.ts:38, test/bench/src/select-legs.workflow.ts:52 -
cell-architecture sandwich-phase-order violate (#8 rejected) test/bench/src/run-bench.service.ts:142-147, test/bench/src/run-bench.service.ts:186-199 -
cell-architecture scoped-lifecycle-boundaries comply test/bench/src/command.ts:34, test/bench/src/command.ts:37 -
cell-architecture service-and-layer-boundaries pre-existing test/bench/src/main.ts:269, test/bench/src/run-bench.service.ts:288 #8 violate, fixed in 2344be12
cell-architecture single-namespace-barrel comply test/e2e-core/src/mod.ts:1-3 -
boundary-testing pin-dependency-semantics violate (#9 confirmed, escalated (E7)); fixed in 6e7a210c test/bench/src/plan-legs.ts:102, test/bench/src/bench-plan.schema.ts:72 #9 violate, rejected
boundary-testing real-system-oracles comply .github/workflows/bench.yml:68, test/bench/src/plan-legs.ts:53 -
boundary-testing staged-protocol-evidence comply test/bench/src/run-path.schema.ts:50 -
schema-laws arbitrary-filter-floors comply test/bench/src/leg-clock.schema.ts:11, test/checker-parity/src/Parity.schema.ts:279 -
schema-laws cross-field-invariants-as-struct-checks violate (#10 rejected) test/bench/src/plan-run-path.workflow.ts:31 -
schema-laws data-only-schema-classes violate (#11 confirmed, fixed in 5f437647) test/bench/src/bench-corpus.schema.ts:5 #10 violate, fixed in 58f6cc3d
schema-laws invariants-as-refinements comply test/bench/src/leg-clock.schema.ts:11, test/e2e-core/src/bench-report.schema.ts:105 -
schema-laws law-failure-is-a-codec-defect comply test/bench/src/bench-plan.schema.ts:72 -
schema-laws refusals-beside-generated-laws comply test/bench/src/leg-clock.schema.ts:47 -
schema-laws rich-type-over-foreign-encoded violate (#12 confirmed, fixed in 66086e27) test/bench/src/bench-plan.schema.ts:41 #11 violate, fixed in 58f6cc3d
schema-laws tagged-unions-over-state-by-presence comply test/e2e-core/src/bench-report.schema.ts:25-28 #12 violate, fixed in feb50851
schema-laws tests-own-no-schemas comply test/bench/src/__tests__/choose-base.workflow.property.test.ts:7 -
package-topology condition-branch-agreement comply test/e2e-core/src/mod.ts:3, test/e2e-core/tsdown.config.ts:6 -
package-topology enumerated-entry-modules comply test/e2e-core/src/mod.ts:3, test/e2e-core/src/mod.ts:1-27 -
package-topology import-time-inertness comply test/e2e-core/src/bench-report.schema.ts:146, test/e2e-core/src/bench-run.schema.ts:77 -
package-topology public-signature-types comply test/e2e-core/src/bench-report.schema.ts:25, test/e2e-core/src/bench-summary.schema.ts:105 -
package-topology surface-changes-are-versioned comply test/e2e-core/src/mod.ts:3, test/e2e-core/src/bench-run.schema.ts:33 -
ci-wall-clock flake-outputs-build-once-on-main comply .github/workflows/bench.yml:32, .github/workflows/bench.yml:26 -
ci-wall-clock heavy-lanes-state-their-daily-minutes comply .github/workflows/bench.yml:1 -
ci-wall-clock heavy-suites-judge-every-main-commit comply .github/workflows/bench.yml:35 -
ci-wall-clock one-stuck-job-backstop comply .github/workflows/bench.yml:42, test/bench/src/plan.ts:113-117 #13 violate, fixed in 2ec2aeb5
ci-wall-clock pr-gate-runs-only-what-the-diff-affects comply .github/workflows/bench.yml:26, test/bench/src/plan-legs.ts:102 -

Not matched by the diff. These rules' applies_when reaches no changed line:

  • cell-architecture: pipeable-dual-parity, resource-vs-handle-duality, staged-lawful-builders, store-serializable-unit-of-work, store-unit-of-work-handle
  • boundary-testing: expected-values-never-come-from-the-subjects-output, fake-and-real-store-laws, no-mocks-on-internal-glue, port-fakes-pass-the-port-contract
  • schema-laws: recursive-schema-suspend
  • package-topology: declared-entry-points, one-access-path
  • mutation-discipline: kill-survivors-with-contract-properties, suppress-by-shape-in-an-ignorer, unkillable-survivors-are-dead-code
  • ci-wall-clock: gates-share-one-job, light-checks-run-the-pinned-tool, no-dev-shell-on-verdict-lanes, no-lane-grades-the-repos-own-docs, red-main-reverts-its-culprit-same-day, release-promotes-the-staging-proven-build, setup-under-a-minute

c0cf821b rows now ruled not matched: none.

Findings and dispositions. The merge produced 12 findings: 7 from the persona lenses and 5 from pack violations. The validator confirmed 6 and rejected 2; #1, #4, #5 and #7 were not validated.

# Sev Finding Location From Validator Disposition
1 P2 Branch head commit scope e2e-core breaks the scope law docs/plans/2026-10-13-0711-ci-bench-off-pr-plan.md:32 project-standards not validated (owner: root) Answered in a888bb61: the plan's scope-law bullet names feb50851 beside 0db0ffa8. Rewriting either subject needs a force push, and the squash merge carries only the PR title ci(bench)!.
2 P3 KD13 boundary claim false: new exports added to e2e-core test/e2e-core/src/bench-summary.schema.ts:98 project-standards confirmed Fixed in a888bb61: KD13 now names the exports the pack fixes added (LegTargetMs, LegTime, RegressedMetric, Regression, PhaseRegressed).
3 P2 Run-path move-back wiring has no test or executed evidence test/bench/src/run-bench.service.ts:189 testing confirmed Escalated as E8: the admission gate forbids unit tests of an operation shell, and the bench has no published surface for an integration test, so the renames' evidence stays the A/A dispatch (AE7). Closed by root: A/A run 38375500452 on 9b518ba7 is the executed evidence; no in-source exception; the abortRunPath laws keep the failure-path restore.
4 P2 Stream-contract change pins side A and reds every push until a leg-free run test/bench/src/plan-legs.ts:53 adversarial not validated (advisory, owner: root) Root ruling (E10): accepted known limit, no fix. The plan already lists it (a commit that changes the stream contract gets no comparison).
5 P2 Burst stacking: per-SHA groups plus 60-minute caps hold 240 runner-minutes .github/workflows/bench.yml:36 adversarial not validated (advisory, owner: root) Root ruling (E10): per-SHA groups stay because every main SHA must be judged; each leg job is capped at 20 minutes. Fixed in cf39161d.
6 P3 Second stderr-tail helper and budget in test/bench test/bench/src/command.ts:24 maintainability confirmed Fixed in f13b9920: command.ts owns the one 4096-character budget and exports tailOf.
7 P3 Generic command runner imports per-program failure schema test/bench/src/command.ts:5 maintainability not validated (over the cap of 8) Report-only. BenchSetupFailed lives in prepare-side.schema.ts, which predates this PR.
8 P1 Sandwich phase order violated: hand-sequenced I/O without Sandwich.named wrapper test/bench/src/run-bench.service.ts:142 pack: cell-architecture rejected Rejected: run-bench.service.ts:176-207 atRunPath is an uninterruptibleMask bracket around a subprocess run whose decision is already a Workflow.make (plan-run-path.workflow.ts:54), with no Encoded command input and no read/decode/encode phases for Sandwich.named to type, and the package's taxonomy keeps cells in *.cell.ts files of which test/bench has none, so the rule's applies_when does not reach the cited line.
9 P1 Pin dependency semantics: no test for Turbo query affected --packages output shape test/bench/src/plan-legs.ts:102 pack: boundary-testing confirmed Escalated as E7. Fixed in 6e7a210c (root's E7 ruling): scripts/turbo-affected-packages.test.ts pins the three facts through the plan job's own Turbo arguments and TurboAffectedPaths. CI has executed it on the exact head 227773f4: run 38416467229, check job 115365565303, ran ./bin/deno test -P --config=scripts/deno.json scripts/, executed 3 tests from scripts/turbo-affected-packages.test.ts (all ok), and ended `ok
10 P2 Cross-field invariants should be on struct, not in consumer code test/bench/src/plan-run-path.workflow.ts:31 pack: schema-laws rejected Rejected: plan-run-path.workflow.ts:26-40 rebased is not a consumer guard before trusting a decoded value: it is the workflow's own decision, and its comparison produces the rebased path the decision returns while failing with the modeled RunPathOutsideTree variant that abort-run-path.workflow.ts:30-31 maps to run-path-move-failed, so moving it to an S.Struct check would convert a decision outcome into a decode refusal and lose that error channel.
11 P1 Schema class used instead of plain data struct test/bench/src/bench-corpus.schema.ts:5 pack: schema-laws confirmed Fixed in 5f437647: the corpus schemas are S.Structs.
12 P1 GreenRunsJson chain not rooted at Schema.*, so it has no generated laws test/bench/src/bench-plan.schema.ts:41 pack: schema-laws confirmed Fixed in 66086e27: GreenRunsJson is rooted at S.Array(...), and ∀x_GreenRunsJson_=x runs.

The lenses recorded 22 residual risks and 14 testing gaps (review.json). Those not already in Known limits or the plan's Risks: the plan job's push path (gh listing, git ancestry, Turbo) first runs on the first push to main; per_page=100 caps the green-run listing, which falls back to the first parent with the full corpus; and a change only to turbo.json, patches/** or .npmrc is reported by Turbo as root-only, so it selects no leg.

Delta correctness lens: f13b9920..aabe4c2a (this round)

One correctness lens, run id 20261013-095812-e6ed3e89, reviewed exactly f13b992031bfb939aead64de88144a239a644235..aabe4c2a22a9ea0dcc929457fc4aff728ea7e804: 64 files, of which the PR's own commits are 6e7a210c (#9), cf39161d (E10) and a0c3d693 (plan); the merges 9b518ba7 and aabe4c2a bring in main. Artifacts: /tmp/compound-engineering-0/ce-code-review/20261013-095812-e6ed3e89/correctness.json, correctness.compact.json and served-models.json. It ran on correctness-reviewer-minimax; its session record shows MiniMax M3.1 (minimax-code/MiniMax-M3.1-Flash-Preview) for all 41 assistant turns: no 429, no fallback, and no call to xai-oauth or Grok. It found the turboAffectedArgs refactor produces an argv byte-identical to the previous inline literal, and TurboAffectedPaths moved verbatim.

# Sev Finding Location Disposition
1 P2 The new Deno pin fails the CI scripts lane without -P scripts/deno.json:6 Accepted by root's ruling: the workflow owner adds -P at ci.yml:137 in a separate root-owned PR, and this PR does not edit ci.yml (E7). Root merges #316 only after that PR is on main and the three pin names appear in the Repo scripts tests log on the merged-main head. Resolved: #338 (cec409b1) is in head 227773f4, and run 38416467229's Repo scripts tests step shows all three pins ok (Evidence).
2 P3 Package description still claims a 60-minute only kill test/bench/package.json:5 Confirmed. Fixed in 4646f883 (also the 0954 plan's caps bullet, R13 and an admission row). Pushed in the merge-up push that made head 227773f4.

Residual risks it recorded: the pins run the repo's Turbo 2.10.12 against a scratch workspace declaring pnpm@9.15.0, so a Turbo or pnpm bump that changes the facts fails loudly; the pins cover direct file changes, not dependent propagation (DependencyChanged), which selectLegs also consumes; the TurboAffectedPaths move was checked by search. Testing gaps: turboAffectedArgs' argument order is checked only through the pin's run, and nothing forces plan-legs.ts to keep using the shared helper.

Range correctness lens: a359a82d..f13b9920

One correctness lens, run id 20261013-081023-0696f63c, reviewed exactly a359a82de4eb1d1627b6ec983435861cfaee40cc..f13b992031bfb939aead64de88144a239a644235 (119 files, including main merged in through e972b7da; the lens judged this PR's own commits). Artifacts: /tmp/compound-engineering-0/ce-code-review/20261013-081023-0696f63c/correctness.json, correctness.compact.json and served-models.json. It ran on correctness-reviewer-minimax. Its session record shows MiniMax M3.1 (minimax-code/MiniMax-M3.1-Flash-Preview) for all 32 assistant turns: no 429, no fallback, and no call to xai-oauth or Grok.

Result: 0 findings. It checked every consumer of the renamed shapes (parentSha → baseSha, PhaseRegression → Regression, LegTime as S.TaggedUnion), the reportOf signature change in main.ts, the cause-carrying run-path rework, and that abortRunPath stays exhaustive with a NEXT_ACTION per code. Residual risks it recorded: leg selection rests on Turbo's query affected semantics (E7); per_page=100 without pagination; a green run whose bench job was skipped becomes a future side A, by design; the ::error escaping was checked by reading. Testing gaps: no end-to-end test of planLegs against real gh/Turbo output (E7), the failWith annotation and writeWallTime are unasserted, and the non-main baseSha sentence of the regression block is not pinned.

Fixes for the final receipt's confirmed findings

b2fe30db # Finding Fixing commit
#2 KD13 says no e2e-core export was added a888bb61 (plan 0749)
#6 Two stderr-tail helpers f13b9920
#11 Corpus schemas are data classes 5f437647
#12 GreenRunsJson loses its generated laws 66086e27
#1 (not validated) Two commit subjects scoped e2e-core a888bb61 (answered in the plan; no force push)
#9 Turbo --packages pin 6e7a210c fix(bench): pin Turbo's affected-packages output in the repo scripts lane (E7)
#3 atRunPath renames' evidence closed by root with A/A run 38375500452 (E8); no code change
#5 (advisory) Burst stacking under 60-minute leg caps cf39161d ci(bench): give each Bench leg job a 20-minute timeout (E10)

After these commits, test/bench passes tsc -b, oxlint, lint:tsgo and 107 tests (18 files), and ∀x_GreenRunsJson_=x runs. A plan dispatch on the head decodes the real corpus and selects all four legs, and a failing gh yields an annotation with the last 4096 characters of its output.

c0cf821b review of 5a088c60: confirmed findings and their fixes

Run id 20261013-063533-c0cf821b, artifacts in /tmp/compound-engineering-0/ce-code-review/20261013-063533-c0cf821b/. That review stopped after validation (KD20), because its fixes move the head; the final receipt above reviews the head that holds them. Every lens started on MiniMax M3.1 and continued on DeepSeek V4.1 Flash after a 429; both validator batches stayed on MiniMax M3.1.

# Sev Finding From Validator Fixing commit
1 P3 Package description claims a first-parent base the lane no longer uses correctness confirmed 377814dd
2 P2 Push-path side A selection (planLegs) has no test testing confirmed e42ae9f6 (plan 9e053e77)
3 P2 Package description still claims first-parent side A project-standards confirmed 377814dd
4 P3 Plan's KTD2 names a run-path service file the change never creates project-standards confirmed 920a29cc
5 P2 Plan job prints an unescaped workflow-command annotation maintainability confirmed a31b3f41
6 P3 Bench package description names the wrong side A rule maintainability confirmed 377814dd
7 P2 Regression bar sits inside the lane's own A/A noise on every leg adversarial rejected rejected: summarize-bench.workflow.ts:203-207 already requires non-overlapping side ranges plus a 10ms/3% floor; no inspected evidence (no A/A spread measurement, no test or data in the reviewed head) shows the bar sits inside the lane's noise, making this an unsupported statistical preference rather than a demonstrated defect.
8 P1 Reserved *.service.ts extension used without Context.Service declaration (pack: cell-architecture, service-and-layer-boundaries.md) pack: cell-architecture confirmed 2344be12
9 P1 Third-party Turbo dependency semantics documented in prose only (pack: boundary-testing, pin-dependency-semantics.md) pack: boundary-testing rejected rejected: The premise 'documented in prose only' is contradicted by the code: plan-legs.service.ts:69-76 enforces the clean-checkout assumption executably via git status --porcelain plus a BenchSetupFailed, and plan-legs.service.ts:88-90 pins Turbo's JSON shape with the TurboAffectedJson decode that fails the step, so no unverified vendor assumption remains in a comment.
10 P1 Schema classes used for plain data instead of S.Struct (pack: schema-laws, data-only-schema-classes.md) pack: schema-laws confirmed 58f6cc3d
11 P1 Foreign Turbo payload kept as domain type instead of decoded to rich type (pack: schema-laws, rich-type-over-foreign-encoded.md) pack: schema-laws confirmed 58f6cc3d
12 P1 Schema state carried by NullOr instead of tagged union (pack: schema-laws, tagged-unions-over-state-by-presence.md) pack: schema-laws confirmed feb50851
13 P1 Plan job does not report wall time to step summary (pack: ci-wall-clock, one-stuck-job-backstop.md) pack: ci-wall-clock confirmed 2ec2aeb5

Escalations to root (all closed by root's resolution of 2026-10-13; plan KD22)

Superseded full-path receipt: head b30908ce, range 25120175..b30908ce, 32 changed files

The review is one ce-code-review run on the full path (mode:agent depth:full base:origin/main), report-only, run id 20261013-043732-5519a046. Its artifacts are in /tmp/compound-engineering-0/ce-code-review/20261013-043732-5519a046/: review.json, metadata.json, stages.jsonl, the per-lens JSON, the pack slices, validator-verdicts.json and served-models.json. All 12 reviewers and both validator batches returned, and none timed out. b30908ce is the remote branch tip; CI, Nix, Changeset Check and Commitlint passed on it. Nothing was rerun.

Verdict from ce-code-review: Ready with fixes. Every pack violation the a359a82d receipt confirmed is fixed by a pushed commit (table below) and is now ruled comply. One pack violation remains confirmed, #7 (heavy-suites-judge-every-main-commit, side A is HEAD^1 rather than the lane's last green commit). It was escalated to root as E3, and root chose the rule's scope (KD16). #7 is fixed in 6a86a164 (side A is the lane's last green main commit, affected legs only), refined by e42ae9f6 (chooseBase: the newest green ancestor, never the pushed commit). The next full-path receipt, run 20261013-071850-b2fe30db on feb50851, rules the rule comply, citing bench.yml:35 and the ancestor check at test/bench/src/plan-legs.ts:67. bench.yml, plan.ts, plan-legs.ts, choose-base.workflow.ts and select-legs.workflow.ts are unchanged from feb50851 to f13b9920, so that ruling holds on the head. Four persona findings (one P2, three P3) are confirmed and listed for root.

Served model per lens. Each model comes from that subagent's session record: its model_change events and the provider/model on each assistant message. No call went to xai-oauth or Grok. No lens tried MiniMax this round. Every reviewer and validator started on DeepSeek V4.1 Flash, with resolvedModelIsFallback: false, because the operator profile (/opt/omp-profile/omp-config.yml, task.agentModelOverrides) maps correctness-reviewer, generalist-reviewer and adversarial-reviewer straight to DeepSeek. That breaks the MiniMax-first rule, and it is escalated as E5.

Lens omp agent Served (assistant turns) Final answer from
correctness correctness-reviewer DeepSeek V4.1 Flash (63) DeepSeek V4.1 Flash
testing correctness-reviewer DeepSeek V4.1 Flash (43) DeepSeek V4.1 Flash
reliability correctness-reviewer DeepSeek V4.1 Flash (35) DeepSeek V4.1 Flash
project-standards generalist-reviewer DeepSeek V4.1 Flash (19) DeepSeek V4.1 Flash
maintainability generalist-reviewer DeepSeek V4.1 Flash (36) DeepSeek V4.1 Flash
adversarial (in-process) adversarial-reviewer DeepSeek V4.1 Flash (38) DeepSeek V4.1 Flash
packs: cell-architecture generalist-reviewer DeepSeek V4.1 Flash (22) DeepSeek V4.1 Flash
packs: boundary-testing + mutation-discipline generalist-reviewer DeepSeek V4.1 Flash (15) DeepSeek V4.1 Flash
packs: schema-laws generalist-reviewer DeepSeek V4.1 Flash (19) DeepSeek V4.1 Flash
packs: package-topology generalist-reviewer DeepSeek V4.1 Flash (23) DeepSeek V4.1 Flash
packs: ci-wall-clock generalist-reviewer DeepSeek V4.1 Flash (12) DeepSeek V4.1 Flash
learnings: docs/solutions generalist-reviewer DeepSeek V4.1 Flash (34) DeepSeek V4.1 Flash
validator batch A (#3, #6, #7) correctness-reviewer DeepSeek V4.1 Flash (15) DeepSeek V4.1 Flash
validator batch B (#1, #2, #4, #5) correctness-reviewer DeepSeek V4.1 Flash (16) DeepSeek V4.1 Flash

The finish leaves are not lenses, and they decide no verdicts. They ran on docs-engineer, which the operator profile serves on Claude Haiku 4.5 (E4, still open): merge leaf (Claude Haiku 4.5 (81)), report leaf (Claude Haiku 4.5 (56)).

Corrections made by the dispatch context. The merge leaf returned unnumbered findings, so they were numbered 1-7 in merge order and attributed to their source lens. finish-input.json had carried scope.head_sha, pr.head_ref_oid, scope.files, scope.diff and the pack directories over from the a359a82d run. All were corrected after the leaves ran, and review.json's and metadata.json's head_sha were set to b30908ce (the correction is recorded in metadata.json). The lens and validator prompts used this run's directory and head. The merge leaf only merged lens returns, and the report leaf only rendered run-directory artifacts. The dispatch stage has a start entry but no end entry in stages.jsonl.

The cross-model peer did not run, because its configured route is xai-oauth/grok. The adversarial lens ran in-process instead.

Packs. Graded from systemfsoftware/systemfsoftware main 763fd663. The upstream main is unchanged since the last receipt:

  • declared in .compound-engineering/config.yaml: cell-architecture, boundary-testing and schema-laws;
  • named by root but not declared: package-topology, mutation-discipline and ci-wall-clock.

Every rule file was ruled: 54 rules in all. 21 are matched by the diff (16 comply, 2 violate, 3 pre-existing) and 33 are not matched.

Pack Rule Ruling Evidence (HEAD file:line) a359a82d receipt
cell-architecture decode-never-cast comply test/e2e-core/src/read-bench-run.workflow.ts:102 -
cell-architecture four-channel-contracts comply test/bench/src/run-path.schema.ts:68 #8 violate, fixed in 26c352b9
cell-architecture pure-decision-workflows comply test/bench/src/measure-leg.workflow.ts:11 -
cell-architecture scoped-lifecycle-boundaries comply test/bench/src/run-bench.service.ts:78 -
cell-architecture service-and-layer-boundaries pre-existing test/bench/src/prepare-side.service.ts:559, test/bench/src/run-bench.service.ts:292 -
boundary-testing no-mocks-on-internal-glue comply test/bench/src/__tests__/settle-run-path.workflow.property.test.ts:6, test/bench/src/run-bench.service.ts:21 -
schema-laws arbitrary-filter-floors comply test/bench/src/leg-clock.schema.ts:11 -
schema-laws data-only-schema-classes comply test/bench/src/run-path.schema.ts:10, test/bench/src/measure-leg.workflow.ts:13 #9 violate, fixed in 08eff64a
schema-laws invariants-as-refinements comply test/e2e-core/src/bench-report.schema.ts:88 #10 violate, fixed in f501c320
schema-laws refusals-beside-generated-laws comply test/e2e-core/src/bench-run.schema.ts:77 #12 violate, fixed in 486ae9b4
schema-laws tagged-unions-over-state-by-presence comply test/e2e-core/src/bench-run.schema.ts:53 #11 violate, fixed in 26c352b9
schema-laws tests-own-no-schemas comply test/bench/src/__tests__/plan-run-path.workflow.property.test.ts:9 -
package-topology enumerated-entry-modules pre-existing test/e2e-core/src/mod.ts:6, test/e2e-core/src/bench-summary.schema.ts:107, test/e2e-core/src/bench-report.schema.ts:25 -
package-topology public-signature-types comply test/e2e-core/src/bench-summary.schema.ts:121, test/e2e-core/src/bench-report.schema.ts:25, test/e2e-core/src/mod.ts:5-7 -
package-topology import-time-inertness comply test/e2e-core/src/bench-report.schema.ts:146, test/e2e-core/src/bench-report.schema.ts:23 -
package-topology surface-changes-are-versioned violate (#6, validator: rejected) test/e2e-core/src/bench-run.schema.ts:33 #13 rejected
ci-wall-clock flake-outputs-build-once-on-main comply .github/workflows/bench.yml:91 -
ci-wall-clock heavy-lanes-state-their-daily-minutes comply .github/workflows/bench.yml:1, .github/workflows/bench.yml:7-8, .github/workflows/bench.yml:11-12 #14 violate, fixed in 913fcbf9
ci-wall-clock heavy-suites-judge-every-main-commit violate (#7, validator: confirmed) .github/workflows/bench.yml:78 #15 violate, fixed in 7d962fca; #7 (scope clause) fixed in 6a86a164 and e42ae9f6, comply on feb50851
ci-wall-clock one-stuck-job-backstop pre-existing .github/workflows/bench.yml:35, .github/workflows/bench.yml:54 -
ci-wall-clock pr-gate-runs-only-what-the-diff-affects comply .github/workflows/bench.yml:19-22 -

Not matched by the diff. These rules' applies_when reaches no changed line:

  • cell-architecture: callable-vs-resource-syntax, handle-state-privacy, pipeable-dual-parity, pipeline-composition, resource-vs-handle-duality, sandwich-phase-order, single-namespace-barrel, staged-lawful-builders, store-serializable-unit-of-work, store-unit-of-work-handle
  • boundary-testing: expected-values-never-come-from-the-subjects-output, fake-and-real-store-laws, pin-dependency-semantics, port-fakes-pass-the-port-contract, real-system-oracles, staged-protocol-evidence
  • schema-laws: cross-field-invariants-as-struct-checks, law-failure-is-a-codec-defect, recursive-schema-suspend, rich-type-over-foreign-encoded
  • package-topology: condition-branch-agreement, declared-entry-points, one-access-path
  • mutation-discipline: kill-survivors-with-contract-properties, suppress-by-shape-in-an-ignorer, unkillable-survivors-are-dead-code
  • ci-wall-clock: gates-share-one-job, light-checks-run-the-pinned-tool, no-dev-shell-on-verdict-lanes, no-lane-grades-the-repos-own-docs, red-main-reverts-its-culprit-same-day, release-promotes-the-staging-proven-build, setup-under-a-minute

Rows that changed ruling without a matching code change: red-main-reverts-its-culprit-same-day (last round violate and rejected) and callable-vs-resource-syntax, real-system-oracles and staged-protocol-evidence (last round comply) were ruled not matched this round. These are the reviewers' readings of applies_when, and nothing was edited to obtain them.

Findings and dispositions. The merge produced 7 findings: 5 from the persona lenses and 2 from pack violations. The validators confirmed 5 and rejected 2.

# Sev Finding Location From Validator Disposition
1 P2 Run-path move/run/move-back wiring has no test test/bench/src/run-bench.service.ts:184 testing confirmed Confirmed, open for root. The plan's testing law refuses a service test, and the move/run/move-back wiring is exercised only by a live Bench run (A/A 38297580066). The last receipt's validator rejected the same finding (#3 there) on that ground; this validator confirmed it on fact. Root decides between a stub-CLI service test and the A/A evidence (E6). Fixed in 26f90637 (every run-path abort decided in a lawed workflow).
2 P3 Corpus empty-repo refusal lost its only test test/e2e-core/src/bench-corpus.schema.ts:16 testing confirmed Confirmed, open for root (E6). S.NonEmptyArray holds the refusal and the generated codec laws draw only accepted values. A refusal block in bench-corpus.schema.ts would be new code in test/e2e-core outside this PR's refinements, which KD13 excludes. Fixed in 8689134a (the schema moved into test/bench/src with its refusal block).
3 P0 bench.yml edited without a CONST-E9 waiver .github/workflows/bench.yml:20 project-standards rejected Rejected: the plan's CONST-W3 declaration (docs/plans/2026-10-13-0435-ci-bench-off-pr-plan.md:41) names CONST-E9 and root's grant.
4 P3 Two near-identical verdict-traversal helpers replace one generic test/e2e-core/src/render-bench-report.workflow.ts:94 maintainability confirmed Confirmed, P3 style. Not fixed in that round: root asked for pack violations only (E6). Fixed in c26ac0ab (one verdict walk).
5 P3 Report fallback hand-rebuilds every BenchReport field test/bench/src/main.ts:264 maintainability confirmed Confirmed, P3 style. Not fixed in that round, for the same reason as #4 (E6). Fixed in 5683035f (the fallback uses the report builder).
6 P1 Package surface changes require versioning test/e2e-core/src/bench-run.schema.ts:33 learnings-package-topology rejected Rejected: test/e2e-core/package.json is "private": true, the same ruling as #13 last round (KD12).
7 P1 Heavy suite scope deviation: bench compares first parent, not lane's last green commit .github/workflows/bench.yml:78 learnings-ci-wall-clock confirmed Confirmed. Escalated as E3 (root's KD1/KD14 choice of HEAD^1 against the rule's last-green scope). Fixed in 6a86a164, refined by e42ae9f6 (root's ruling KD16); comply on feb50851.

The residual risks the lenses recorded are already listed in Known limits, the plan's Risks or the settled decisions: a 60-minute kill leaves no report; the enterprise registry install (layer 2); a burst of merges stacks legs; and bench.yml's environment is verified only by a live run.

Fixes for the b30908ce receipt's confirmed findings

b30908ce # Finding Fixing commit (subject) In head f13b9920
#1 Run-path wiring has no test 26f90637 fix(bench): decide every run-path abort in a lawed workflow yes
#2 Corpus empty-repo refusal lost its test 8689134a fix(bench): move the corpus schema into test/bench and state its refusals yes
#4 Two verdict-traversal helpers c26ac0ab fix(bench): walk the phase verdicts once in the report renderer yes
#5 Report fallback hand-rebuilt 5683035f fix(bench): build the report-unwritable fallback with the report builder yes
#7 heavy-suites-judge-every-main-commit, scope clause, bench.yml:78 6a86a164 fix(bench): bench a push from the lane's last green main commit, affected legs only; refined by e42ae9f6 fix(bench): choose side A in a lawed decision, the newest green ancestor yes

Each mapping comes from the fixing commit's own message, which names the b30908ce row it fixes. The rejected rows keep their reasons above: #3 (CONST-W3 declaration names CONST-E9) and #6 (test/e2e-core is private, KD12).

The clause #7 violated, quoted from heavy-suites-judge-every-main-commit.md: "a push run tests the diff from the lane's last green main commit to this commit, after checking that commit is an ancestor of this one. The full corpus runs only on a workflow_dispatch that names no jobs, when the lane's own inputs change (its workflow file, its config, its tool pin), and as the fallback when no green ancestor is found, so a run never under-tests." On f13b9920: side A is needs.plan.outputs.base (bench.yml:106-110), chosen by chooseBase as the newest green push run whose SHA passes git merge-base --is-ancestor and is not the pushed commit; selectLegs runs the full corpus on a dispatch, when no green ancestor exists, and when the lane's inputs changed; there is no schedule. bench.yml has no dispatch inputs, so every dispatch names no jobs and runs the full corpus.

Fixes for the a359a82d receipt's confirmed violations

a359a82d # Rule Fixing commit (subject) Ruling on b30908ce In head f13b9920
#8 four-channel-contracts 26c352b9 fix(bench): carry run-path settle outcomes as tagged causes comply yes
#9 data-only-schema-classes 08eff64a fix(bench): make the report's leg time plain data comply yes
#10 invariants-as-refinements f501c320 fix(bench): refine the leg's elapsed time and the report's ref comply yes
#11 tagged-unions-over-state-by-presence 26c352b9 (same commit as #8) comply yes
#12 refusals-beside-generated-laws 486ae9b4 fix(bench): state the refusal boundary of every refinement this PR adds comply yes
#14 heavy-lanes-state-their-daily-minutes 913fcbf9 ci(bench): state the lane's measured runner-minutes in its header comply yes
#15 heavy-suites-judge-every-main-commit 7d962fca ci(bench): give each dispatch its own concurrency group (dispatch-group clause); the last-green scope clause (E3) fixed later in 6a86a164 (KD16) violate on b30908ce (scope clause, E3); comply on feb50851 yes

Each mapping is taken from the fixing commit's own message, which names the a359a82d row it fixes. Validator-rejected rows of that receipt keep their reasons: #13 (surface-changes-are-versioned, the package is private, KD12).

Escalations raised by the b30908ce receipt (E3, E5 and E6 resolved by root on 2026-10-13: KD16, KD19 and KD17; E4 stays open above)

Superseded full-path receipt: head a359a82d, range 25120175..a359a82d, 32 changed files

The review is one ce-code-review run on the full path (mode:agent depth:full base:origin/main), report-only, with run id 20261013-030833-9213d365. Its artifacts are in /tmp/compound-engineering-0/ce-code-review/20261013-030833-9213d365/: review.json, metadata.json, stages.jsonl, the per-lens JSON, the pack slices, validator-verdicts.json and served-models.json. All 12 reviewers and both validator batches returned, and none timed out.

Verdict from ce-code-review: Not ready. Seven pack violations are confirmed and escalated to root below (E1–E4). Root's 2026-10-13 merge-cycle intent names exactly one blocker for this cycle: placement under the no-new-test/e2e-core rule, plus this final-head receipt. The escalated violations are not part of that blocker. No fix work has started on them.

Served model per lens. Each model comes from that subagent's session record: its model_change events and the provider/model on each assistant message. No call went to xai-oauth or Grok. Every lens started on MiniMax, which returned 429 ("Token Plan rate limit reached"). The harness then switched to its configured DeepSeek fallback and recorded resolvedModelIsFallback: true.

Lens omp agent Served (assistant turns) Final answer from
correctness correctness-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (52) DeepSeek V4.1 Flash
testing correctness-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (40) DeepSeek V4.1 Flash
reliability correctness-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (32) DeepSeek V4.1 Flash
project-standards generalist-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (22) DeepSeek V4.1 Flash
maintainability generalist-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (40) DeepSeek V4.1 Flash
adversarial (in-process) adversarial-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (34) DeepSeek V4.1 Flash
packs: cell-architecture generalist-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (21) DeepSeek V4.1 Flash
packs: boundary-testing + mutation-discipline generalist-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (26) DeepSeek V4.1 Flash
packs: schema-laws generalist-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (27) DeepSeek V4.1 Flash
packs: package-topology generalist-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (21) DeepSeek V4.1 Flash
packs: ci-wall-clock generalist-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (11) DeepSeek V4.1 Flash
learnings: docs/solutions generalist-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (17) DeepSeek V4.1 Flash
validator batch A (9 P1) correctness-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (16) DeepSeek V4.1 Flash
validator batch B (7 persona) correctness-reviewer MiniMax M3.1 (1), then a 429 and DeepSeek V4.1 Flash (28) DeepSeek V4.1 Flash

The finish leaves are not lenses. They decide no verdicts: the validators judged every finding. They ran on docs-engineer (see E4):

  • merge leaf, three runs (Claude Haiku 4.5 (30), Claude Haiku 4.5 (33), Claude Haiku 4.5 (33)). Run 1 dropped the ci-wall-clock rulings and the pack findings. Run 2 dropped the persona findings. Both were rejected and are kept as *.rejected-1/-2. Run 3 is the merge of record. Its finding docs(global): rewrite root README for stryker-js-effect #8 location (bench.yml:52) was corrected to test/bench/src/run-path.schema.ts:52, from the slice evidence.
  • report leaf (Claude Haiku 4.5 (78)).

The cross-model peer did not run, because its configured route is xai-oauth/grok. The adversarial lens ran in-process instead.

Packs. The cached pack resolution was stale (02bf9653, 2026-10-10) and had no ci-wall-clock pack. All packs were therefore graded from systemfsoftware/systemfsoftware main 763fd663:

  • declared in .compound-engineering/config.yaml: cell-architecture, boundary-testing and schema-laws;
  • named by root but not declared: package-topology, mutation-discipline and ci-wall-clock;
  • storybook-testing and ui-state also exist upstream, but no UI or story file is in the diff.

Every rule file was ruled: 54 rules in all. 21 are matched by the diff (11 comply, 9 violate, 1 pre-existing) and 33 are not matched.

Pack Rule Ruling Evidence (HEAD file:line) Basis
cell-architecture callable-vs-resource-syntax comply test/bench/src/measure-leg.workflow.ts:25, test/bench/src/run-path.schema.ts:7 new deciders are Workflow.make values; the new tagged classes stay non-callable records
cell-architecture decode-never-cast comply test/bench/src/run-bench.service.ts:130, test/bench/src/plan-run-path.workflow.ts:11 the RUNNER_TEMP listing reaches planRunPath through a decoded command, with no cast
cell-architecture four-channel-contracts violate (#8, validator: confirmed) test/bench/src/run-path.schema.ts:52, test/bench/src/run-bench.service.ts:167, test/bench/src/prepare-side.schema.ts:6 RunPathUnrestored stores Cause.pretty text in restoreError/runFailure instead of cause: S.optional(S.Unknown) (sibling BenchSetupFailed has it)
cell-architecture pure-decision-workflows comply test/bench/src/measure-leg.workflow.ts:28, test/bench/src/main.ts:142 measureLeg declares error: S.Never; time is read in the shell and passed in the command
cell-architecture scoped-lifecycle-boundaries comply test/bench/src/run-bench.service.ts:173 move-back runs under Effect.uninterruptibleMask, so teardown survives interruption
cell-architecture service-and-layer-boundaries pre-existing test/bench/src/prepare-side.service.ts:16, test/bench/src/main.ts:317 driver imports in *.service.ts are on unchanged lines
boundary-testing no-mocks-on-internal-glue comply test/bench/src/__tests__/measure-leg.workflow.property.test.ts:11, test/bench/src/run-bench.service.ts:176, test/bench/src/run-bench.service.ts:82 no vi.fn/vi.mock/stand-in FileSystem; laws drive pure decisions
boundary-testing real-system-oracles comply test/bench/src/run-bench.service.ts:173 the lane runs the real CLI on real corpora; renames hit the real filesystem
boundary-testing staged-protocol-evidence comply test/bench/src/run-path.schema.ts:7, test/bench/src/leg-clock.schema.ts:10, test/bench/src/run-bench.service.ts:170 move-in/run/move-back is typed: atRunPath needs a RunPathMoveIn, settle yields RunPathRestored/RunPathUnrestored
schema-laws arbitrary-filter-floors comply test/e2e-core/src/bench-report.schema.ts:27, test/e2e-core/src/bench-summary.schema.ts:110, test/e2e-core/src/__tests__/render-bench-report.workflow.property.test.ts:434 refinements use built-in checks (NonNegativeFinite, NonEmptyArray), so derived arbitraries need no filter
schema-laws data-only-schema-classes violate (#9, validator: confirmed) test/e2e-core/src/bench-report.schema.ts:26, test/e2e-core/src/bench-report.schema.ts:33, test/bench/src/measure-leg.workflow.ts:27 LegMeasured/LegNotMeasured are TaggedClasses with a family TypeId, but no Workflow.make decides LegTime since the decision moved to LegClock
schema-laws invariants-as-refinements violate (#10, validator: confirmed) test/bench/src/leg-clock.schema.ts:6, test/e2e-core/src/bench-report.schema.ts:27, packages/stryker-js-plugin-interface/src/Metrics.schema.ts:20 LegElapsed.wallMs is bare S.Finite (the >= 0 rule lives in the decider); report ref is bare S.String
schema-laws tagged-unions-over-state-by-presence violate (#11, validator: confirmed) test/bench/src/settle-run-path.workflow.ts:13, test/bench/src/run-path.schema.ts:52, test/e2e-core/src/bench-report.schema.ts:75 SettleRunPathCommand.restoreError/runFailure are NullOr and the decider branches on presence
schema-laws tests-own-no-schemas comply test/bench/src/__tests__/settle-run-path.workflow.property.test.ts:13, test/bench/src/measure-leg.workflow.ts:6 the new laws draw production schemas; no test declares its own schema
schema-laws refusals-beside-generated-laws violate (#12, validator: confirmed) test/bench/src/run-path.schema.ts:9, test/e2e-core/src/bench-summary.schema.ts:110, test/e2e-core/src/bench-report.schema.ts:21 refined fields (run-path.schema.ts NonEmptyStrings; e2e-core metrics, nextAction) have no in-source refusal property; precedent at test/e2e-core/src/annotation.schema.ts:107
package-topology surface-changes-are-versioned violate (#13, validator: rejected) test/e2e-core/src/bench-run.schema.ts:31, test/e2e-core/src/bench-report.schema.ts:94 narrowed exports of test/e2e-core with no changeset
ci-wall-clock heavy-lanes-state-their-daily-minutes violate (#14, validator: confirmed) .github/workflows/bench.yml:1, .github/workflows/bench.yml:4-6 bench.yml has no header comment pricing runner-minutes per day (the figures are in this PR body only)
ci-wall-clock heavy-suites-judge-every-main-commit violate (#15, validator: confirmed) .github/workflows/bench.yml:12, .github/workflows/bench.yml:68 a dispatch shares bench-${{ github.sha }} instead of github.run_id; side A is HEAD^1 with the full corpus every push, not the last green commit with an affected-only run
ci-wall-clock one-stuck-job-backstop comply .github/workflows/bench.yml:19, .github/workflows/bench.yml:38 both jobs at timeout-minutes: 60, in-process deadline deleted
ci-wall-clock pr-gate-runs-only-what-the-diff-affects comply .github/workflows/bench.yml:4-6 the PR trigger and hand-kept paths-ignore are removed
ci-wall-clock red-main-reverts-its-culprit-same-day violate (#16, validator: rejected) .github/workflows/bench.yml:144, .github/workflows/bench.yml:153 no P0/culprit filing on a red main run; the only red-path change is always() on uploads

Not matched by the diff. These rules' applies_when reaches no changed line:

  • cell-architecture: handle-state-privacy, pipeable-dual-parity, pipeline-composition, resource-vs-handle-duality, sandwich-phase-order, single-namespace-barrel, staged-lawful-builders, store-serializable-unit-of-work, store-unit-of-work-handle
  • boundary-testing: expected-values-never-come-from-the-subjects-output, fake-and-real-store-laws, pin-dependency-semantics, port-fakes-pass-the-port-contract
  • schema-laws: cross-field-invariants-as-struct-checks, law-failure-is-a-codec-defect, recursive-schema-suspend, rich-type-over-foreign-encoded
  • package-topology (named by root; not declared in config): condition-branch-agreement, declared-entry-points, enumerated-entry-modules, import-time-inertness, one-access-path, public-signature-types
  • mutation-discipline (named by root; not declared in config): kill-survivors-with-contract-properties, suppress-by-shape-in-an-ignorer, unkillable-survivors-are-dead-code
  • ci-wall-clock (named by root; not declared in config): flake-outputs-build-once-on-main, gates-share-one-job, light-checks-run-the-pinned-tool, no-dev-shell-on-verdict-lanes, no-lane-grades-the-repos-own-docs, release-promotes-the-staging-proven-build, setup-under-a-minute

Findings and dispositions. The merge produced 16 findings: 7 from the persona lenses and 9 from pack violations. The validators confirmed 7 and rejected 9.

# Sev Finding Location From Validator Disposition
1 P2 Cascade: hung registry install now stacks 60-minute legs per main push .github/workflows/bench.yml:38 adversarial rejected Rejected by the validator. The single 60-minute kill is R6/AE5 and the settled KD2; the enterprise install is layer 2 (KD9).
2 P2 Enterprise registry install unbounded below the 60-minute job kill test/bench/src/prepare-side.service.ts:505 reliability rejected Rejected by the validator. The cited line is unchanged, and the missing bound is the deliberate R6 removal; layer 2 owns the install.
3 P2 Run-path abort branches have no test test/bench/src/run-bench.service.ts:197 testing rejected Rejected by the validator. The mapped decisions are lawed, and the plan refuses a service test (the testing law; A/A run 38297580066 is the evidence).
4 P2 New code added to test/e2e-core against root's scope law test/e2e-core/src/bench-summary.schema.ts:106 project-standards rejected Rejected by the validator. BenchSummary.regression is the plan's named carve-out. Root still decides the scope-law reading; see E2.
5 P2 Total verdict ignores an unrecorded reporting phase test/e2e-core/src/summarize-bench.workflow.ts:103 correctness rejected Rejected by the validator. Reporting reads not-recorded only on verdict lines from before 19.0.0, where mutation-test already absorbed reporting time, so total stays comparable.
6 P3 Effect.reduce accumulator where Effect.forEach supplies the index test/bench/src/run-bench.service.ts:289 maintainability rejected Rejected by the validator: an equivalent style change (Effect 4.0.0's reduce passes the index).
7 P3 Two new render helpers duplicate one verdict walk test/e2e-core/src/render-bench-report.workflow.ts:83 maintainability rejected Rejected by the validator: the two walks select different arms and emit different text.
8 P1 four-channel-contracts.md violation: Errors must carry cause, not pretty-printed strings test/bench/src/run-path.schema.ts:52 learnings confirmed Confirmed. Escalated as E1; a fix is ready in test/bench only.
9 P1 data-only-schema-classes.md violation: LegTime class family has no decision contract test/e2e-core/src/bench-report.schema.ts:26 learnings confirmed Confirmed. Escalated as E2; the fix edits test/e2e-core.
10 P1 invariants-as-refinements.md violation: LegElapsed.wallMs lacks domain refinement test/bench/src/leg-clock.schema.ts:6 learnings confirmed Confirmed. Escalated: E1 for LegElapsed, E2 for e2e-core ref.
11 P1 tagged-unions-over-state-by-presence.md violation: settle leg uses nullable fields for state test/bench/src/settle-run-path.workflow.ts:13 learnings confirmed Confirmed. Escalated as E1; a fix is ready in test/bench only.
12 P1 refusals-beside-generated-laws.md violation: New refinements lack refusal properties test/bench/src/run-path.schema.ts:9 learnings confirmed Confirmed. Escalated: E1 for test/bench, E2 for e2e-core.
13 P1 surface-changes-are-versioned.md violation: Export changes not recorded in changeset test/e2e-core/src/bench-report.schema.ts:94 learnings rejected Rejected by the validator. The rule covers published packages, and test/e2e-core/package.json:3 is "private": true.
14 P1 heavy-lanes-state-their-daily-minutes.md violation: Bench lane has no cost documentation .github/workflows/bench.yml:1 learnings confirmed Confirmed. Escalated as E3: the header has to price the run shape root chooses.
15 P1 heavy-suites-judge-every-main-commit.md violation: Dispatch and push share concurrency group .github/workflows/bench.yml:12 learnings confirmed Confirmed. Escalated: E1 for the dispatch group, E3 for last-green and affected-only.
16 P1 red-main-reverts-its-culprit-same-day.md violation: Red postsubmit has no failure handling .github/workflows/bench.yml:144 learnings rejected Rejected by the validator: applies_when is not reached, since no failure handler is added. Root's KD3 (report-only) stands.

Residual risks the lenses recorded are all already in Known limits, the plan's Risks, or settled decisions:

  • a 60-minute kill leaves no report;
  • the stream-contract mismatch;
  • the registry install (layer 2);
  • per-SHA fan-out;
  • an abort drops finished runs;
  • run identity across sides is asserted by the A/A run, not checked by the lane;
  • test/bench is not in the mutation lane (Stream K).

Escalations raised by that receipt (E1 and E2 resolved by root on 2026-10-13; see Pack compliance round)

Earlier rounds (superseded)

These were reviews of individual lenses, not full-path. Their dispositions carry forward and were re-judged above:

  • f1da499b, 63ba7bb9 and a359a82d: six lenses each, served by MiniMax with DeepSeek fallback. They drove 63ba7bb9, the enterprise cwd-equals-tree draw, and b9ebfbb5, which names both causes in run-path-move-failed.
  • e5844df9: MiniMax with DeepSeek fallback.

The passes on f6b3ec88, c8372c03, eec5cfa5 and the plan included xai-oauth/grok-4.7, so under Ryan's rule they count as unreviewed. The fixes they drove (e684c838, eec5cfa5, b91e981a) are on this head.

Mutation law scope (root resolution, 2026-10-13)

Ryan's nightly-only mutation law covers the repository's mutation-testing quality lanes, mutation.yml and kill-matrix.yml, which Stream K owns (#289 aligns with K). It does not cover Bench. Bench is a speed measurement of the product over fixtures, like the e2e lane, which also runs on every push to main. So bench.yml stays on push to main plus workflow_dispatch and compares each main SHA with the lane's last green main commit. This PR edits neither mutation workflow, and no evidence here waits on a Mutation run of main. Any later mutation evidence will come from the next nightly run or from a dispatch root authorizes.

Bench cost (heavy-postsubmit rule)

Every figure below comes from GitHub Actions job timestamps (completed_at − started_at, summed over jobs) for the 7 days 2026-10-06T01:54Z to 2026-10-13T01:54Z: 2,471 runs and 11,718 jobs that ran. Nothing was rerun. Bench has not yet run on a push to main, so per-push cost is taken from the seven workflow_dispatch runs of this design: two sides, eight runs per leg, no in-process deadline.

Per leg (the old plan job and four legs, all ubuntu-latest; every one of these runs benched the full corpus)

Run Plan vitest-runner typescript-checker enterprise e2e-core Job-minutes First start to last end
38297580066 (this PR's design, head 896a494c) 0.07 5.05 7.68 3.83 3.67 20.30 10.8 min
38279943505, 38144176368, 38143012318, 38140987766, 38138985254, 38135504932 (#312 branch) 0.07–0.12 3.08–4.80 6.58–7.18 2.35–3.30 1.95–3.23 15.33–17.68 6.7–9.2 min

Across all seven runs, a push costs 15.3–20.3 job-minutes (median 17.0). The jobs of 38297580066 are 114972296814, 114974586413, 114974586425, 114974586457 and 114974586517. GitHub's timing API reports 0 billable ms for that run, because this public repository runs on standard runners. So the cost is runner occupancy, not billed minutes.

Per day

  • Main took 110 pushes in the window, or 15.71 per day. That is 110 Release push runs on main, matching the 110 first-parent commits in git log.
  • The real plan program, run over each of the 110 commits with its parent as side A (as if every run were green), picks no leg for 16, the enterprise leg alone for 2 and all four legs for 92, 64 of them because the lane's own inputs changed. The new plan job costs 0.40–0.71 min: the same setup steps as timed in the bench jobs above, plus the plan step itself (2.7–3.8 s, timed locally).
  • Priced with each leg's median job-minutes, that week is 1,655 job-minutes, about 236 per day. With each leg's minimum and maximum the range is 1,333–1,948, or 190–278 per day. The full corpus on every push would be 271 per day at the medians.
  • Today's PR-path Bench used 500.7 job-minutes per day (3,504.7 over 7 days, 194 pull_request runs). This PR therefore cuts Bench's runner use by about 220–310 job-minutes per day.

Compared with Mutation (same window, job-minutes per run)

  • Push-triggered Mutation: median 152.6, mean 161.7, max 813.0 (90 runs that ran jobs).
  • Scheduled nightly Mutation: median 62.5, range 43.5–161.5 (6 runs).
  • One Bench run is about 0.11–0.13 of a median push-triggered Mutation run and 0.25–0.32 of a median nightly.
  • Mutation run cost spreads widely: the cheapest observed runs were 31.0 job-minutes (push) and 34.5 (dispatch), against Bench's 15.3–20.3.
  • Per day the order reverses once K makes Mutation nightly-only. Bench's 190–278 job-minutes per day would be about 3.0–4.4 times one nightly Mutation run at its median.

Can Bench starve PR CI? Partly: when the runner pool is already busy.

  • The org's concurrent-job limit is not visible to this token; GET /orgs/systemfsoftware returns no plan field. The observed behaviour stands in for it:

    • ubuntu-latest peaked at 88 concurrent jobs (2026-10-10T22:51Z).
    • Jobs with no needs start within a 2 s median while fewer than 40 jobs are running.
    • With 40–49 running, their queue median is 33 s; with 50–59, 127 s; with 60–69, 224 s.
    • Over the week, 445 PR CI / check jobs queued for a median of 2 s, with a 90th percentile of 401 s.

  • Bench holds at most 4 runners per push, and the plan job ends before any bench job starts. Each leg runs 2.0–7.7 minutes, and each SHA has its own concurrency group, so a burst of merges stacks legs.

  • Replaying the measured week with PR Bench removed and a full-corpus run added at each observed main-push time (a model; it assumes the added jobs start without queueing). The affected-only selection only removes jobs from it:

    Scenario Hours ≥ 40 concurrent Hours ≥ 50 concurrent Peak
    Observed 11.98 6.91 88
    With a full-corpus run per push 11.45 6.00 83

  • Net: Bench can add queue time to PR CI only when the pool is already at about 40 or more jobs, and this PR lowers that total load compared with today.

Not measured: a Bench run on an actual push to main (the first comes after merge); the org's configured concurrency limit; the queue cost of jobs that needs another job (their created_at is not their queue time). Raw data is in session scratch under /tmp/bench-cost/.

Known limits

@systemfsoftware-maker
ci(bench)!: bench each main commit against its first parent
f6b3ec8 
The bench leaves the pull-request path. bench.yml runs on push to main
(docs-only pushes excluded) and on workflow_dispatch, with concurrency
bench-${{ github.sha }} and no cancel, so no PR pays for or waits on it.

- Side A is the first parent, side B the pushed commit; both are worktrees
  under RUNNER_TEMP with their own released tarballs and frozen install,
  and the checkout holds only the harness.
- Every in-process deadline is gone: BENCH_DEADLINE_MS, the setup budget,
  the run deadline, BENCH_RUN_TIMEOUT_MS, and the setup-timed-out,
  budget-exceeded, timed-out and timeout shapes. Every timeout-minutes is
  60; that job timeout is the only kill.
- The report carries the leg's wall time (from JOB_STARTED_AT) against a
  10-minute target, never enforced, and schemaVersion moves to 2.0.
- Only a leg's total decides a regression (non-overlapping samples and
  |delta| >= max(3% of A, 10 ms)). The regression block names the commit,
  its parent, total then every regressed phase, and the next action: a
  revert PR or root-fix P0 on main, speed evidence only elsewhere.
- Every run executes from RUNNER_TEMP/bench-run, so both sides record the
  same project root, inputs digest and mutant ids. Bench configs reference
  plugins relative to the config file so they survive the move.

Plan: docs/plans/2026-10-12-1800-ci-bench-off-pr-plan.md (layer 1; the
enterprise install is layer 2).

BREAKING CHANGE: bench-report.json schemaVersion 2.0 drops the
setup-timed-out and budget-exceeded abort codes and the timed-out run exit,
and adds ref, leg and the regression block.
@systemfsoftware-maker
docs(bench): say an environment-incomplete abort writes no report
e684c83 
Review finding (correctness, P3): the README said every aborted leg uploads
its report, but main.ts publishes environment-incomplete without writing
bench-report.json or the step summary.
@systemfsoftware-maker
ci(bench): drop the hand-written path filter from the main trigger
17e21b4 
Root amendment 2026-10-12: no change classifier where Turbo can do the job.
paths-ignore (**/*.md, docs/**) skipped none of 11 sampled main commits;
turbo query affected skips three of them. Every push now benches every
leg until layer 1b scopes legs with turbo query affected.

Plan: docs/plans/2026-10-12-1854-ci-bench-off-pr-plan.md supersedes the
1800 plan and records the turbo-query evidence and rule-4 reasons.
@systemfsoftware-maker
Merge remote-tracking branch 'origin/main' into stryker/bench-off-pr
896a494
@systemfsoftware-maker
docs(plan): bench every main SHA; defer Turbo affected-only leg selec…
498f871 
…tion

Sub-conductor ruling 2026-10-12: the acceptance is a comparison per main
SHA, so Turbo's affected-packages answer stays evidence, not a leg filter.
The 1926 plan supersedes 1854: layer 1b is dropped, KD10 keeps Turbo as
the owner of builds and cache and adds no classifier, KD11 defers
affected-only selection to root.
@systemfsoftware-maker
Merge remote-tracking branch 'origin/main' into stryker/bench-off-pr
c8372c0
@systemfsoftware-maker
fix(bench): name phase-only slowdowns instead of claiming no phase ch…
eec5cfa 
…anged

A summarized report with no regression said "no project/phase changed past
the signal threshold" even when its table showed a phase that regressed under
an unchanged total. The notice is now titled "Bench phase-only slowdown", says
no leg total crossed the regression threshold, and lists each such cell as a
phase-only slowdown. A new law drives a check-only slowdown through
summarizeBench and renderBenchReport.

The plan moves to docs/plans/2026-10-12-2032-ci-bench-off-pr-plan.md
(supersedes 1926) and adds the CONST-W3 declaration for the bench.yml edits,
the corrected cause of the stream-undecodable PR runs (#298 changes the
stream contract), and that case as a known limit.
@systemfsoftware-maker
docs(plan): list every bench.yml edit in the CONST-W3 waiver and scop…
b91e981 
…e the PR-run evidence

Review of eec5cfa found that the waiver called side B the checkout (it is the bench-b worktree) and left out the checkout ref, Turbo cache and harness install edits. Run 38284821646 also had a budget-exceeded abort on one leg, so the Problem Frame now scopes stream-undecodable to the other three legs.
@systemfsoftware-maker
test(e2e-core): move the bench decision checks beside their workflows…
0db0ffa 
… as laws

bench-decisions.integration.test.ts fed hand-written NDJSON lines and
report text to the pure readBenchRun and coveringTestFiles workflows and
asserted their decision fields. Under root's 2026-10-12 testing law an
integration test drives the real product and asserts only observable
outcomes, and a pure decision gets property laws beside it (CONST-T12,
CONST-T14). This layer had to change the file (ReadBenchRunCommand's
exit field), so it is deleted rather than carried.

The behaviours no other suite pinned are now laws over the production
schemas: testsExecuted sums every drawn MutantCost, the first undecodable
line is named by its 1-based number among blank and decodable lines, a
verdict count other than one is stream-invalid, verdict phase durations
are carried as reported or refused when absent, covering test files are
the sorted distinct holders, and a non-JSON report is refused. The
legacy-line decode stays pinned by the cli-contract wire-line suite.

The plan moves to docs/plans/2026-10-12-2154-ci-bench-off-pr-plan.md
(supersedes 2032) with the widened edit scope and test admission rows.
@systemfsoftware-maker
Merge remote-tracking branch 'origin/main' into stryker/bench-off-pr
e5844df
@ryanleecode ryanleecode mentioned this pull request Oct 13, 2026
Closed
@systemfsoftware-maker
refactor(bench): keep the bench's new decisions in test/bench
e28351f 
Root's 2026-10-13 scope law puts no new code in test/e2e-core. The
layer's four new modules and their laws move to test/bench/src, which is
their only consumer: run-path.schema, plan-run-path and settle-run-path
(the fixed run path) and measure-leg (the leg wall time). e2e-core's
mod.ts is back to main.

measureLeg now decides a bench-local LegClock (LegElapsed or
LegClockBehind) that main.ts maps onto the report's LegTime:
make-body-purity forbids a decision from constructing a class imported
from another package, and e2e-core cannot import test/bench.

test/bench gains a test lane shaped like test/checker-parity: a test
script, a vitest config with inSourceSchemaLaws, tsconfig.test.json and
lint:tsgo over it. The plan moves to
docs/plans/2026-10-13-0223-ci-bench-off-pr-plan.md (supersedes 2154).
@systemfsoftware-maker
Merge remote-tracking branch 'origin/main' into stryker/bench-off-pr
f1da499
@systemfsoftware-maker
fix(bench): abort an outside-tree run path as a defect
63ba7bb 
Review of f1da499 (MiniMax lenses, report-only) found two defects in
the bench's run path:

- planRunPath's identity law drew cwd and CLI tails of length one or
  more, so the enterprise shape (cwd equal to the tree) was never
  generated. Dropping the `path === tree` operand left the law green;
  with empty tails allowed it now fails.
- A cwd or CLI outside its side's tree aborted as run-path-move-failed,
  whose next action says to fix the runner. It is a bench bug, so it
  now aborts as defect. The README lists that case.

The plan moves to docs/plans/2026-10-13-0238-ci-bench-off-pr-plan.md
(supersedes 0223) with an accurate split of the e2e-core law files.
@systemfsoftware-maker
fix(bench): name both causes in the run-path-move-failed action
b9ebfbb 
63ba7bb sent an outside-tree run path to `defect`. Review of that head
found two problems: it contradicted KTD3, and `defect`'s next action asks
for a stack trace, which a typed failure does not have. So the outside-tree
case aborts as run-path-move-failed again. That code's next action, which
this layer added, now names both causes. An OS error needs a runner fix; a
path outside the tree is a bug in side setup. The README matches.

The plan moves to docs/plans/2026-10-13-0258-ci-bench-off-pr-plan.md
(supersedes 0238), with U4 brought back in line with KTD3.
@systemfsoftware-maker
Merge remote-tracking branch 'origin/main' into stryker/bench-off-pr
a359a82
@systemfsoftware-maker
docs(plan): plan the pack-violation fixes for the bench lane
90df094 
Supersedes the 0258 plan. Root (2026-10-13) requires every confirmed pack
violation in the a359a82 full-path receipt to be fixed, not waived.
Adds KD12-KD15 and units U7-U12, the e2e-core boundary (one in-source
refusal block per refined e2e-core file, LegTime as a data union), and
test-admission rows for the refusal properties and the settle law.
@systemfsoftware-maker
fix(bench): carry run-path settle outcomes as tagged causes
26c352b 
Fixes full-path receipt findings #8 (four-channel-contracts) and #11
(tagged-unions-over-state-by-presence) on a359a82.

The settle command took restoreError and runFailure as nullable
Cause.pretty strings and decided on their presence. It now takes two
tagged unions, RunPathMoveBack (MovedBack | MoveBackFailed { cause }) and
RunPathRunEnd (RunSucceeded | RunFailed { cause }), and matches on the
move-back. RunPathUnrestored carries cause: S.optional(S.Unknown) and the
run's end instead of strings; the service passes each Exit's Cause as is.
Text is rendered once, in the error's message, when the report's reason
is built.

The settle law draws the production SettleRunPathCommand and asserts
unrestored iff the move-back failed, with both causes carried by
identity. Plan: docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U7.
@systemfsoftware-maker
fix(bench): make the report's leg time plain data
08eff64 
Fixes full-path receipt finding #9 (data-only-schema-classes) on a359a82.

LegMeasured and LegNotMeasured were TaggedClass variants carrying a
LegTime type id that no Workflow.make checks; LegTime is only a field of
BenchReport. LegTime is now an S.TaggedUnion of the same two cases, so
bench-report.schema.ts declares two fewer classes and no type id, and the
encoded shape is unchanged. main.ts and the render law construct through
LegTime.cases.

Evidence: tsc, oxlint and lint:tsgo clean in test/e2e-core and
test/bench; vitest 364/364 in test/e2e-core (the 3 fewer tests are the
generated codec laws of the two deleted classes; LegTime's own laws
remain) and 39/39 in test/bench. Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U8.
@systemfsoftware-maker
fix(bench): refine the leg's elapsed time and the report's ref
f501c32 
Fixes full-path receipt finding #10 (invariants-as-refinements) on
a359a82, both fields named by root.

- LegElapsed.wallMs was S.Finite, so a negative elapsed time decoded as
  the "clock advanced" variant. It is now finite and >= 0, with the
  invariant stated in the check's message.
- BenchReport.ref was S.String while SummarizeBenchCommand.ref is
  S.NonEmptyString. It is now S.NonEmptyString, and main.ts reads
  BENCH_REF with Config.NonEmptyString, so an empty value takes the
  existing environment-incomplete path (main.ts program, UNKNOWN_REF)
  instead of failing at BenchReport.make.

Evidence: tsc, oxlint and lint:tsgo clean in test/e2e-core and
test/bench; vitest 364/364 and 39/39. The refusal properties for these
refinements land with #12. Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U9.
@systemfsoftware-maker
fix(bench): state the refusal boundary of every refinement this PR adds
486ae9b 
Fixes full-path receipt finding #12 (refusals-beside-generated-laws) on
a359a82.

The generated schema laws draw only accepted values. Each refined schema
the PR adds or changes now states its refusals in its own file, in one
import.meta.vitest block whose subject is a private predicate. Each
block draws the unrefined base, appends boundary values to every draw,
and compares acceptance with a predicate written from the domain:

- test/bench: run-path.schema.ts (all ten path fields), leg-clock.schema.ts
  (elapsed ms finite and >= 0), and the commands of plan-run-path,
  settle-run-path and measure-leg.
- test/e2e-core, the narrowest arrangement under root's scope law (plan
  KD13: no new file, export, dependency, config or helper):
  bench-report.schema.ts (LegTargetMs, ref, abort codes = the codes with a
  next action, rendered exit 0|1), bench-summary.schema.ts (RegressedMetric
  and PhaseRegression fields), bench-run.schema.ts (failure codes without
  timeout, exitCode a safe integer, never null) and summarize-bench (ref).

Evidence: tsc, oxlint and lint:tsgo clean in both packages; vitest
44/44 in test/bench (+5) and 375/375 in test/e2e-core (+11). The pack's
widen-once check is not run: root forbids hand-applied probes. Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U10.
@systemfsoftware-maker
ci(bench): give each dispatch its own concurrency group
7d962fc 
Fixes full-path receipt finding #15 (heavy-suites-judge-every-main-commit)
on a359a82.

The group was bench-${{ github.sha }} for both triggers, so a
workflow_dispatch on main's head shared, and could replace, the pending
push run for that commit. A push now groups by github.sha and a
dispatch by github.run_id, with cancel-in-progress false, as the rule's
Concurrency bullet requires. Why side A is HEAD^1 rather than the last
green commit is stated in the lane's cost header (#14). Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U11.
@systemfsoftware-maker
ci(bench): state the lane's measured runner-minutes in its header
913fcbf 
Fixes full-path receipt finding #14 (heavy-lanes-state-their-daily-minutes)
on a359a82.

bench.yml now opens with the cost header the rule asks for:
- the daily figure, about 267 runner-min/day (241-319), priced on the
  run every push to main starts;
- that run's shape, the full corpus against the first parent, and why
  side A is HEAD^1 rather than the last green commit (the corpus is a
  fixed workload; an older base selects no smaller workload and would
  credit several pushes' change to one commit), which also answers #15;
- the per-run cost, 15.3-20.3 job-min, with the seven run ids;
- the 7-day first-parent merge count, 110 in 2026-10-06T01:54Z to
  2026-10-13T01:54Z (15.71/day);
- whether the lane can starve PR CI, from the same week's job
  timestamps: only once about 40 jobs are already running; the replay
  without the pull_request Bench lowers hours at 40+ jobs from 11.98 to
  11.45.

Every figure is from GitHub Actions job timestamps already reported in
the PR's Bench cost section; nothing was rerun. Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U12.
@systemfsoftware-maker
docs(plan): record the landed concurrency, settle law and header shape
9fe1e38 
Supersedes the 0416 plan. R3, U1 and the CONST-W3 list now carry the
push-sha/dispatch-run-id concurrency group from 7d962fc. U7's test
sentence matches the landed settle law (it draws the production command
and asserts the carried causes, not message text, whose oracle would
copy Cause.pretty). U12 records that the HEAD^1 rationale lives in the
cost header.
@systemfsoftware-maker
docs(plan): say where the HEAD^1 rationale lives in U11
b30908c 
Supersedes the 0434 plan. U11 no longer claims an inline comment; the rationale is in the cost header (U12).
@systemfsoftware-maker
docs(plan): plan the fixes for every confirmed row of the b30908c rec…
4fac41f 
…eipt

Supersedes the 0435 plan. Root's merge bar (2026-10-13) requires a fixing
commit on the reported head for every confirmed pack violation and accepted
finding. Adds KD16 (push runs scoped from the last green main commit, legs
selected by Turbo's affected set), KD17-KD19 and units U13-U18.
@systemfsoftware-maker
Merge remote-tracking branch 'origin/main' into stryker/bench-off-pr
2d59a2e
@systemfsoftware-maker
fix(bench): build the report-unwritable fallback with the report builder
5683035 
Fixes b30908c full-path receipt finding #5 (maintainability: the fallback
hand-rebuilt every BenchReport field). reportOf now takes report runs, and
writeReport's fallback calls it with the report's own fields.

Plan: docs/plans/2026-10-13-0554-ci-bench-off-pr-plan.md U13.
@systemfsoftware-maker
fix(bench): walk the phase verdicts once in the report renderer
c26ac0a 
Fixes b30908c full-path receipt finding #4 (maintainability: two
near-identical verdict walks). One traversal yields each improved or slower
phase change; the improved and phase-only lists filter it. Only lines this
PR added change.

Plan: docs/plans/2026-10-13-0554-ci-bench-off-pr-plan.md U14.
@systemfsoftware-maker
fix(bench): move the corpus schema into test/bench and state its refu…
8689134 
…sals

Fixes b30908c full-path receipt finding #2 (testing: the corpus empty-repo
refusal lost its only test when bench-decisions.integration.test.ts was
deleted).

bench-corpus.schema.ts is consumed only by the bench, so it moves from
test/e2e-core/src to test/bench/src, unchanged, and e2e-core's mod.ts drops
its export. Beside it, one in-source refusal block states every refinement
the corpus carries: each text field refuses only the empty string, and each
list (repo, mutate, testFiles) refuses only the empty list. The repo list's
refusal is the empty-repo case the deleted test covered; the bench aborts
such a corpus as corpus-unreadable.

test/bench: 54 tests (44 + 2 refusal properties + 8 generated codec laws);
test/e2e-core: 367 (375 less the moved schema's 8 codec laws).

Plan: docs/plans/2026-10-13-0554-ci-bench-off-pr-plan.md U15 (amended in
the next plan revision: a move, not a workflow).
@systemfsoftware-maker
fix(bench): declare the bench corpus as plain data
5f43764 
Fixes b2fe30db #11 (schema-laws data-only-schema-classes, confirmed):
BenchRepoEntry, BenchEnterpriseCorpus and BenchCorpus were S.Class
declarations, though none is a Workflow.make command, a decision variant
or a tagged error; every consumer only reads their fields. They are now
S.Struct schemas with exported types. BenchCorpusJson keeps its S.*-rooted
chain, so its generated laws still run, and the refusal block is
unchanged.

Plan: docs/plans/2026-10-13-0749-ci-bench-off-pr-plan.md U28.
@systemfsoftware-maker
refactor(bench): keep one stderr-tail helper for the bench's commands
f13b992 
Fixes b2fe30db #6 (maintainability, confirmed): command.ts kept a
4000-character tail over a CommandOutcome, and run-bench.service.ts a
second 4096-character tail over a string, both added by this PR.
command.ts now owns the one budget (4096, private) and exports
tailOf(text); runChecked, the interrupt message and runCli's stderr
collector all use it. A setup failure keeps 96 more characters of
output.

Smoke: a plan run whose gh exits 7 after 6000 bytes of stderr fails with
an annotation that carries the last 4096 characters.

Plan: docs/plans/2026-10-13-0749-ci-bench-off-pr-plan.md U29 (the plan
says both names are exported; only tailOf has an importer, so the budget
stays private).
This was referenced Oct 13, 2026
Merged
Merged
systemfsoftware-maker added a commit that referenced this pull request Oct 13, 2026
@systemfsoftware-maker
docs(plans): record the root's plan review of the test/e2e-core disso…
0ba3a7e 
…lution

The root accepted D1, D2, D3 (guards checked first), D5 and D6, the Done
grep, no-rename, no-shim, R6, R7, the inventory and the landing table.

Its five review corrections are now in the plan:
- no scratch-edit proofs: refusals are committed negative fixtures or a
  closed deliberate-red PR (bench lint ban, E2E-1's missing-journey case);
- a contract codec is never mutation-graded, because Mutation selects
  *.workflow.ts only; PR C's codec is graded by its schema laws;
- #306 and #327 fixed the ManifestDocument and CountedReportJson flakes,
  so the law moves unchanged with no follow-up unit;
- D2's corpus change is timed before the post-#316 speed baseline, or
  recorded as a reviewed corpus change;
- landing order A, then C at any time, B after #316 and #298, D last.

PR A's evidence and its one-time cost (the first main Mutation run after
merge re-verifies every mutant, because the cache path list changes)
are named in the plan.
ryanleecode pushed a commit that referenced this pull request Oct 13, 2026
@systemfsoftware-maker
ci(mutation): grade only the shipped products, not test/e2e-core (#337)
3797deb 
* docs(plans): plan the dissolution of test/e2e-core into the e2e harness, the bench and the CLI contract

Per-file destinations for all 80 files, a four-layer stack (Mutation de-enrolment, bench, stream codec, harness and deletion), Stream K sequencing and the landing order of the open PRs that touch the package. ce-doc-review skipped: the root reviews this plan (r42).

* docs(plans): refresh the test/e2e-core dissolution plan against main and the root's corrections

Applies the root's six corrections of 22:25: MUTATED_PROJECTS is data at lockfile-closure-digest.integration.test.ts:29-34 and changes with mutation PROJECTS; stryker-js-cli-contract is not in PROJECTS, so the moved stream codec is ungraded until Stream K enrols it; the lint coupling at test/bench/oxlint.config.ts:1, test/bench/tsconfig.node.json:11, test/e2e/oxlint.config.ts:17 and test/e2e/AGENTS.md:21; kill-matrix.yml 9/15/82 and mutation.yml 36/43/64/118; the exact inventory (29 src modules plus mod.ts, 13 laws, 7 suites, 18 fixtures, 12 package files); and verified or [unverified]-marked pack and config claims.

Adds Done and Does-not-count, the 28 outside files that name the package by layer, #287's overlap with PR D, #321's merge, and D6 for the three fixture schemas under tests-own-no-schemas.

* docs(plans): record the root's plan review of the test/e2e-core dissolution

The root accepted D1, D2, D3 (guards checked first), D5 and D6, the Done
grep, no-rename, no-shim, R6, R7, the inventory and the landing table.

Its five review corrections are now in the plan:
- no scratch-edit proofs: refusals are committed negative fixtures or a
  closed deliberate-red PR (bench lint ban, E2E-1's missing-journey case);
- a contract codec is never mutation-graded, because Mutation selects
  *.workflow.ts only; PR C's codec is graded by its schema laws;
- #306 and #327 fixed the ManifestDocument and CountedReportJson flakes,
  so the law moves unchanged with no follow-up unit;
- D2's corpus change is timed before the post-#316 speed baseline, or
  recorded as a reviewed corpus change;
- landing order A, then C at any time, B after #316 and #298, D last.

PR A's evidence and its one-time cost (the first main Mutation run after
merge re-verifies every mutant, because the cache path list changes)
are named in the plan.

* ci(mutation): grade only the shipped products, not test/e2e-core

Mutation and the kill matrix mutated four projects: the three product
packages and test/e2e-core, a private test-tooling package. Test tooling
is not shipped, so grading it spends shard time on code no user runs
(plan R7).

- mutation.yml drops test/e2e-core from PROJECTS (:36), from the
  INCREMENTAL_REPORTS cache paths (:43), and from the incremental (:64)
  and build (:118) cache keys.
- kill-matrix.yml drops it from both default project lists (:9, :15)
  and from its build cache key (:82), which stays byte-identical to
  mutation.yml's so the kill matrix still restores the mutation build.
- lockfile-closure-digest.integration.test.ts drops it from
  MUTATED_PROJECTS, the test's copy of Mutation's project list, so its
  live-lockfile scenario checks the projects Mutation actually grades.

The package itself, its mutation script and stryker.config.ts stay until
the last layer deletes the package.

One-time cost: a cache entry's version is a hash of its path list, so
the first main Mutation run after this lands cannot restore the previous
incremental reports and re-verifies every mutant once.

* test(stryker-js): read the mutated projects from mutation.yml in the lockfile test

The lockfile-closure scenarios checked "every mutated project" against a
hard-coded copy of Mutation's corpus, which nothing compared with the
workflow. The test now reads env.PROJECTS from mutation.yml, so the two
lists cannot drift apart (review run 20261013-084434-c89dc2ab, testing P2).

CONST-W3: this PR edits .github/workflows/mutation.yml and kill-matrix.yml,
which AGENTS.md:7 marks read-only and CONST-E9 treats as a judgment surface.
Root assigned that edit when it accepted the plan (PR A). The edit only
removes test/e2e-core from the corpus; no threshold, baseline or check
changes. .stryker/budget-baseline.json stays as measured over four projects,
and its owner decides whether to re-measure it.

Plan 0900 supersedes 0218 with these review outcomes.

* chore(changeset): record a none intent for the lockfile test's corpus change

release.jsonc now gates intents on turbo's typecheck graph (#335), whose inputs include tests, so the test edit in this PR needs an intent. Nothing published changes.

* docs(plans): name the workflow-derived corpus wherever plan 0900 cited MUTATED_PROJECTS

Delta review run 20261013-090355-9173e728 (P3): the Problem Frame, the PR D ordering risk, the landing note and P9 still described the retired constant. They now give it as main's state at 381b3f5 and name the read of mutation.yml env.PROJECTS that replaces it.
@systemfsoftware-maker
fix(bench): pin Turbo's affected-packages output in the repo scripts …
6e7a210 
…lane

Fixes b2fe30db #9 (P1, boundary-testing pin-dependency-semantics, E7):
the plan job's Turbo --packages output was pinned only by a measurement.

scripts/turbo-affected-packages.test.ts runs the plan job's own Turbo
arguments (turboAffectedArgs, now shared with plan-legs.ts) through
node_modules/.bin/turbo, the binary `pnpm exec turbo` resolves to, in
scratch pnpm workspaces, and decodes the output through TurboAffectedPaths,
the schema plan-legs.ts decodes with. It pins three facts: a changed package
reports its repo-relative directory (packages/a), a root file reports "",
and no change reports [].

TurboAffectedPaths moves to turbo-affected.schema.ts: the scripts lane
type-checks with Deno, which cannot resolve bench-plan.schema.ts's in-source
law block (@systemfsoftware/vitest, import.meta.vitest). Its generated codec
laws still run there.

scripts/deno.json takes #319's permission hunk byte for byte, plus an
effect/ import-map entry, and scripts/assert.ts is #319's file byte for
byte, so merging #319 later is a no-op for both. Deno applies the grant
only with -P, which ci.yml:137 does not pass yet; root's workflow owner
adds it in a separate PR. Until then these three tests fail in the Repo
scripts tests step with NotCapable, as #319's do.

Plan: docs/plans/2026-10-13-0954-ci-bench-off-pr-plan.md U31, KD22.
@systemfsoftware-maker
ci(bench): give each Bench leg job a 20-minute timeout
cf39161 
Root's E10 ruling: per-SHA concurrency groups stay, because every main
SHA must be judged, and each leg job is capped at 20 minutes instead of
60. A leg's longest measured job is 7.68 job-minutes. The plan job keeps 60.

Plan: docs/plans/2026-10-13-0954-ci-bench-off-pr-plan.md U32, KD2.
@systemfsoftware-maker
docs(plan): record root's resolution of E3-E11 and the 20-minute leg cap
a0c3d69 
Supersedes the 0749 plan. KD22 records root's rulings of 2026-10-13: #9
fixed by the scripts-lane Turbo pin (U31), inert until ci.yml passes -P in
a separate root-owned PR; E10's 20-minute leg cap (U32, KD2); E8 and E11
closed by A/A run 38375500452; #4 and E4 accepted; E9 satisfied by #335.
@systemfsoftware-maker
Merge remote-tracking branch 'origin/main' into stryker/bench-off-pr
aabe4c2
@systemfsoftware-maker
docs(bench): state the 20-minute leg-job timeout in the package descr…
4646f88 
…iption and plan

Fixes the f13b992..aabe4c2 correctness lens finding #2 (P3, confirmed):
cf39161 lowered each Bench leg job's timeout to 20 minutes, but the
@systemfsoftware/stryker-bench description still said GitHub's 60-minute
job timeout is the only kill, and the 0954 plan's CONST-W3 caps bullet, R13
and the read-bench-run admission row still said 60 minutes. The plan job
keeps 60, so plan.ts's step-summary line is unchanged.

Plan: docs/plans/2026-10-13-0954-ci-bench-off-pr-plan.md U32, KD2.
@systemfsoftware-maker systemfsoftware-maker mentioned this pull request Oct 13, 2026
Merged
@ryanleecode ryanleecode mentioned this pull request Oct 13, 2026
Merged
ryanleecode
ryanleecode approved these changes Oct 13, 2026
View reviewed changes

@ryanleecode ryanleecode left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Root verdict: full-path receipt (feb5085) plus delta lenses f13b992..aabe4c2 and aabe4c2..227773f (MiniMax M3.1, 0 findings) cover the exact head. CI green on 227773f, including the turbo pin tests under -P (run 38416467229, check job 115365565303: 13 passed, 0 failed). 0 open threads, 0 behind main. The open P3 (stale KD22 sentence at plan:101) is plan prose with no behaviour, so I've deferred it to the next plan revision rather than adding a push and a CI re-run.

@ryanleecode
ryanleecode merged commit 77fbae3 into main Oct 13, 2026
17 checks passed
@ryanleecode
ryanleecode deleted the stryker/bench-off-pr branch October 13, 2026 16:22
@systemfsoftware-maker systemfsoftware-maker mentioned this pull request Oct 13, 2026
Open
systemfsoftware-maker added a commit that referenced this pull request Oct 13, 2026
@systemfsoftware-maker
Merge main into the verdict store: stream files out of inputs, bench …
cad54c6 
…off PRs

Takes #329 and #316. No conflicts; #329's always-excluded default stream path sits beside this branch's temp-sibling rule for the incremental file. The pre-push gate refuses a tip behind main, so this lands with the previous merge in one push.
@systemfsoftware-maker systemfsoftware-maker mentioned this pull request Oct 13, 2026
Open
systemfsoftware-maker added a commit that referenced this pull request Oct 13, 2026
@systemfsoftware-maker
Merge remote-tracking branch 'origin/main' into stryker/tsgo-checker-…
8457ce7 
…shortcut

Brings main up to efd1996 (#328 type queries, #322 CheckerRuntime split, #331, #313, #316, #329, #340). Two import conflicts, both additive: type-query.handle.ts keeps main's context-link imports and this branch's shared decided from ts-compiler.handle.ts; compare-sides' law imports both Deferred (main) and DigestCall (this branch). #322 removed CheckerRuntime.layer from ./runtime, so the importer-shortcut and isolated-declarations integration tests now build the runtime with TypeScriptCheckerRuntime.layer.
This was referenced Oct 13, 2026
Merged
Merged
systemfsoftware-maker added a commit that referenced this pull request Oct 13, 2026
@systemfsoftware-maker
build(turbo): declare the bench modules the repo scripts suite imports
299075f 
The owner's merge of main (73cc2e0) brought #316's scripts/turbo-affected-packages.test.ts into //#test:scripts. It imports test/bench/src/turbo-affected.ts and turbo-affected.schema.ts, which were in no input: an edit to them left the task's hash unchanged and --affected selected nothing. Both are now inputs; measured on Turbo 2.10.12, a bench edit moves the hash (96c8c1fd -> 8e13d64d) and selects the task. The plan moves to a new dated file that supersedes the 10:28 one.
This was referenced Oct 13, 2026
Merged
Merged
systemfsoftware-maker added a commit that referenced this pull request Oct 13, 2026
@systemfsoftware-maker
docs(plans): record the pre-U1 Bench baseline for T10
03e7e42 
Root's reading of the first four green main Bench push runs after #316
(38429923569, 38430066932, 38436282117, 38436508210) replaces the plan's
[unverified] baseline run ids, with the per-leg median-of-four phase table.
Those runs read `instrument` as about 0 ms on every leg because the mark is
misplaced, so they are the baseline later T10 layers compare against, not the
T9 reading: R18 now reads `S` from the first four green main push runs after
U1 merges. R17 states Bench's non-overlapping-samples rule, since a single
typescript-checker run took 33.5 s against about 46 s for the other three.
@systemfsoftware-maker systemfsoftware-maker mentioned this pull request Oct 13, 2026
Merged
ryanleecode pushed a commit that referenced this pull request Oct 13, 2026
@systemfsoftware-maker
perf(stryker-js): enter the instrument phase before instrumenting (T1…
b08379d 
…0 U1) (#348)

* perf(stryker-js): enter the instrument phase before instrumenting

The `instrument` phase mark and its `instrument` span ran in the instrument
cells' write phase, after `instrumentFiles` and `makeSandbox` had already run
in the read phase. Instrumentation and the sandbox copy therefore counted
under `prepare`, and every Bench sample read `instrument` as 0-1 ms, so the
instrument share of wall clock that gates the T9 native-core restart could not
be read.

Both cells (`instrument.cell.ts` for runs, `plan-instrument.cell.ts` for
plans) now open the phase span and emit the mark at the start of the read
phase; the write phase only offers file skips. The new engine integration test
drives a dry-run-only run and a plan through the published `Engine` entry
points and asserts from OTel span intervals that the gather and sandbox
acquisition spans fall inside the single `instrument` span and that the phase
mark ends before gathering starts. It failed on the old placement and passes
on the new one.

Lands the T10 infection-filtered execution plan (U1 of it, R18, KTD9).

* docs(plans): record the pre-U1 Bench baseline for T10

Root's reading of the first four green main Bench push runs after #316
(38429923569, 38430066932, 38436282117, 38436508210) replaces the plan's
[unverified] baseline run ids, with the per-leg median-of-four phase table.
Those runs read `instrument` as about 0 ms on every leg because the mark is
misplaced, so they are the baseline later T10 layers compare against, not the
T9 reading: R18 now reads `S` from the first four green main push runs after
U1 merges. R17 states Bench's non-overlapping-samples rule, since a single
typescript-checker run took 33.5 s against about 46 s for the other three.

* test(stryker-js): pin the instrument phase on a failing sandbox build

Review finding (P3, testing): the changeset says a run that fails while
instrumenting or building its sandbox now reports the `instrument` phase as
entered, but no scenario drove that failure. The new scenario writes projects
whose `buildCommand` exits 1 and runs dry-run-only mutation runs in them.
Through `Engine.mutationTestCell` and the stage event queue it asserts the run
fails in the `instrument` stage with reason `Sandbox initialization failed`,
after `prepare` then `instrument` were reported entered. Through
`Engine.strykerCell` and the span exporter, in a second fresh project, it
asserts the same stage failure and that the failed gathering and sandbox work
stay inside the one `instrument` phase span. Against main's cells the phases
read `['prepare']` and no phase span exists; on this branch both hold.

Every scenario runs inside its temp project's working directory, so config
discovery cannot pick up the package's own Stryker config.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ryanleecode ryanleecode ryanleecode approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@systemfsoftware-maker @ryanleecode