Repository navigation
docs(global): rewrite root README for stryker-js-effect - #8
Merged
Merged
Conversation
Replace template content with a dedicated project README documenting the Effect 4 mutation testing framework, streaming NDJSON protocol, exit codes, and package directory
Use pnpm and pnpm exec exclusively across install and execution examples
This was referenced Oct 10, 2026
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 10, 2026
…ery mutant record A mutant's `redundancy` and `readmission` fields become one optional `subsumption` tagged union: `Subsumed` (rule, dominators) only on an Ignored mutant, `Readmitted` (rule, causes) only on a mutant that ran. The plugin Mutant schema, the 8.0 wire line, the incremental report and the reuse records all carry the single key, and a reference that disagrees with its status is refused (review #4, #5). Tests: a refusal law on the plugin Mutant decode, a wire-line scenario for both tags on Ignored and Killed lines, literal status-reason text and the `>=` complement row in the instrumenter (#2, #3), and readmit scenarios for every reachable cause code: ignored at check, compile error, remembered without running, unsettled (#8). A subsumed mutant closed early by ignoreStatic keeps the static reason and no reference, because its dominator is static too (#13).
ryanleecode
pushed a commit
that referenced
this pull request
Oct 10, 2026
* docs(plans): scope unit I mutant subsumption and proved equivalents Requirements-only brainstorm for static subsumption (Layer 1) and z3-solver equivalence proving (Layer 2), with the kill-matrix no-signal-loss predicate and open questions for the supervisor. * docs(plans): plan unit I mutant subsumption implementation Enrich the requirements-only plan with the planning contract: eleven stacked units, the kill-matrix drop audit (R16/R17 predicate made total, Unattested rule verdict), the z3-solver driver outside the CLI bundle, and S3 moved last behind its own drop audit. Apply the report-only ce-doc-review findings: R24 cites SMT-LIB '=' semantics, R29 restated as a sampling net, R2/R4/R13 aligned with KTD2/KTD4, merged report carries statusReason. * docs(plans): replace SMT layer with type-guided generation in unit I plan Apply the root's rulings: drop z3-solver and async-mutex, make Layer 2 type-guided generation from TS 7 checker facts read on the unmutated program, add generic check-time re-admission to PR-A, and regroup into four stacked PRs (PR-A, PR-B, PR-W, PR-C). Fold in the round-2 report-only document review. * fix(stryker-js-instrumenter)!: stop dropping condition-position relational mutants The sufficient-set table Ignored the complement operator and one boolean literal at every relational site in an if, loop, or ternary test. Its proof assumes numeric operands. In if (a < b) with a = null and b = undefined, the dropped true mutant enters the branch and none of the kept a <= b, a != b, false does; operands whose valueOf has side effects defeat the operator drops too. Delete the table's consumers (relationalSufficientReplacement, the relationalSufficient fact, and the policy's redundant-relational rule). Generation is unchanged, so mutant ids are unchanged. Plan U12 (R4, R7; AE4). * feat(stryker-js-instrumenter): decide relational complement subsumption per site subsumeMutants is a pure workflow over serializable site and candidate facts: at a relational site under the default policy, a statically kept complement (>= for <, > for <=, <= for >, < for >=) is Subsumed by every statically kept dominator (<=, <, >=, >). The four ordering operators read both operands once through ToPrimitive(number) and differ only in how they map the one IsLessThan result, so whenever the dominator differs from the original the complement equals the dominator. Nothing calls the workflow yet. Plan U2 (R1, R3, R5, R7, R10, R11). * feat(stryker-js-instrumenter): drop subsumed relational complements with a redundancy reference planMutants decides each candidate's static reason first (directive, excluded mutator, ignorer, mutant-set policy, arid), then asks subsumeMutants which statically kept complements a kept ordering operator dominates. A subsumed mutant stays placed, is Ignored as redundant-relational, and carries a typed redundancy reference whose first dominator id is the kept mutant. The plugin Mutant gains the optional redundancy field (refused unless Ignored) and redundancyStatusReason, which renders the reason code, dominator id and next action. Plan U4, U5 (plugin Mutant part); R3, R5, R7, R8, R9, R11; AE1, AE3. * feat(stryker-js-plugin-interface): name why a subsumed mutant was re-admitted A plugin Mutant gains an optional readmission field: the subsumption rule and, per named dominator, a cause code (ignored at check, compile error, ignored at plan, remembered without running, unsettled) with detail. The decoder refuses it together with a redundancy reference or an Ignored status. Plan U5, U13 (R31, R9); agentic-quality rule. * feat(stryker-js): re-admit subsumed mutants at check time and keep drop references on records A subsumed mutant is held out of the checker and runner until its dominators settle. If none of them runs (ignored at check, compile error, ignored at plan, remembered without running) it is re-admitted and runs, naming each dominator and why; otherwise it settles Ignored naming a dominator that ran. The shard plan keeps it with its first dominator. The run stream moves to 7.0 with redundancy/readmission on every mutant line, the incremental record keeps both, and a record carrying either is re-decided each run under the new decidedPerRun refusal, ranked below every other cause. Plan U5, U13; R5, R9, R31, AE2. * test(e2e): expect stream schema 8.0 on the failing-run terminal record PR-A moves the run stream to 8.0 (merge of #262); the failing-run journey asserts the terminal error record's schemaVersion. * fix(stryker-js-instrumenter): type a candidate's arid reason as ignore-rule vocabulary The candidate schema took any string as aridReason, so main's EveryIgnoreReasonNamesItsRule law (#262) failed whenever a draw carried non-vocabulary arid text. The field is now Mutant.IgnoreStatusReasonText. * refactor(stryker-js): reuse optionalField and derive duplicated lists ce-simplify-code pass: plan-request and mutation-test-plan use the shared optionalField; RUNNING_STATUSES derives from Settled; a readmission takes its rule from the held entry; Mutator.service imports RelationalOperator from its schema file and the re-export goes. * feat(stryker-js-plugin-interface)!: carry one subsumption union on every mutant record A mutant's `redundancy` and `readmission` fields become one optional `subsumption` tagged union: `Subsumed` (rule, dominators) only on an Ignored mutant, `Readmitted` (rule, causes) only on a mutant that ran. The plugin Mutant schema, the 8.0 wire line, the incremental report and the reuse records all carry the single key, and a reference that disagrees with its status is refused (review #4, #5). Tests: a refusal law on the plugin Mutant decode, a wire-line scenario for both tags on Ignored and Killed lines, literal status-reason text and the `>=` complement row in the instrumenter (#2, #3), and readmit scenarios for every reachable cause code: ignored at check, compile error, remembered without running, unsettled (#8). A subsumed mutant closed early by ignoreStatic keeps the static reason and no reference, because its dominator is static too (#13). * fix(stryker-js): keep the subsumption reference through stryker merge The merged JSON report now carries each mutant's `subsumption` reference from the shard streams, so a sharded run reports the same records as an unsharded one. The shard-merge journey adds a relational site and co-locates the subsumed pair in one shard (review #7). * docs: describe complement subsumption and agree on stream schema 8.0 The READMEs and the mutation-testing skill name the relational complement drop and its readmission (review #1). The changesets agree that the stream moves to 8.0, describe the single subsumption union, and state the score change (review #10). * docs: show the 8.0 subsumption key in the machine-stream sample The README's machine-stream sample still carried the removed redundancy and readmission keys, so the documented 8.0 mutant lines did not decode. The cli-contract README now names stream version 8.0. * fix(stryker-js-cli-contract): state the Readmitted rule the codec enforces A Readmitted reference is accepted on any mutant that is not Ignored: a re-admitted plan mutant has no status yet, and a checkpoint writes it as Pending before it settles. The refusal messages and changesets said it must have run, which the codec never checked. * refactor(stryker-js): one present-field helper and a Subsumed-only planning test The merge command builds every optional report key through one presentField helper. The planner strips run fields only from Subsumed mutants, matching the shard placement and early-result paths. * test(e2e): annotate the subsumed relational complements as Ignored Under the default mutant-set policy the `<=` complement of `n > 0` and `count > 0` is Ignored, subsumed by the kept `>=` mutant. The calc, vm-vitest and svelte fixtures still annotated every EqualityOperator mutant Killed, so their oracle scenarios failed on the PR's e2e legs.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 11, 2026
…vious run The previous-run lookup treated any compare API error as 'not an ancestor' and could fall through to E2E_RED_NO_PREVIOUS with a one-commit range. A failed compare now stops the lookup and reports E2E_RED_PREVIOUS_UNKNOWN with gh's error, no range and no candidates. Review finding #8.
ryanleecode
pushed a commit
that referenced
this pull request
Oct 12, 2026
…sts (#314) * docs(plans): e2e postsubmit on main and one 60-minute backstop Requirements-only plan (ce-brainstorm, Lightweight) for moving the e2e legs off pull requests into a push-to-main workflow, setting every timeout-minutes outside bench.yml to 60 and deleting the step-level timeout wrappers. Records the jobs-API before numbers and the answers on parity --deadline, the mutation budget exit codes and exit-124 readers. * ci: run e2e postsubmit on main with one 60-minute backstop Move the five e2e legs out of ci.yml into e2e.yml, which runs on push to main and on workflow_dispatch only, with concurrency e2e-<sha> and no cancel-in-progress, and skips release commits as before. No pull-request workflow runs pnpm test:e2e any more. A non-green run writes one ::error and a step summary with this SHA, the previous e2e run on the same ref (newest completed run whose legs ran on a strict ancestor), its conclusion, the compare range with each commit as a culprit candidate, a reason code (E2E_RED_AFTER_GREEN, E2E_RED_AFTER_RED, E2E_RED_NO_PREVIOUS) and the next action. Each leg writes its wall time against a 10-minute target and warns on an overrun without changing its outcome. Every timeout-minutes outside bench.yml becomes 60. The step-level kills go: the 1200 s wrapper on the E2E lane, the "$BUDGET" wrapper on the checker-parity run step (and its --deadline, which run does not need), and the 300 s install timeout in bake-fixtures.sh. The exit-124 shard-timeout branch goes with its wrapper; an empty exit now reads shard-not-run. mutation.yml's gate step warns instead of failing only when the pinned CLI exits 1 with the BudgetExceeded message; exit 2, any other exit, or exit 1 without that message still fails the job. Plan: docs/plans/2026-10-11-0905-ci-e2e-postsubmit-plan.md (R1-R21). * ci(mutation): note that a reworded budget message fails the gate * ci(e2e): TEMPORARY run e2e on pushes to ci/e2e-postsubmit * test(e2e): TEMPORARY plant a red svelte-app assertion to prove the e2e report * ci(e2e): revert the planted red and the TEMPORARY branch trigger Restores svelte-app.e2e.test.ts's exit-code assertion and drops the ci/e2e-postsubmit push trigger, so e2e.yml runs on push to main and on dispatch only. Proof runs: 38128581338 (green) and 38129591374 (E2E_RED_AFTER_GREEN on the planted leg). mutation.yml's gate step: the budget is the workflow's only verdict and report-only, so the summary says that, and the BUDGET_EXCEEDED next action asks for a per-shard executed vs reused comparison against the previous main run (a reuse-dropping lockfile edit is fixed by #303 once released) before any baseline update. Exit handling unchanged. * ci(checker-parity): drop the unreachable step-cap cause from shard-not-run The job and the run step share a 60-minute cap, so a step-cap timeout cancels the job and the failure() step that writes this annotation never runs. Review finding #7. * ci(e2e): dump container logs when a leg is cancelled by its timeout GitHub cancels a job that reaches timeout-minutes, and failure() is false for a cancelled job, so a hung leg lost its docker logs. The step now also runs on cancelled(). Review finding #6. * ci(e2e): report a failed compare call instead of reading it as no previous run The previous-run lookup treated any compare API error as 'not an ancestor' and could fall through to E2E_RED_NO_PREVIOUS with a one-commit range. A failed compare now stops the lookup and reports E2E_RED_PREVIOUS_UNKNOWN with gh's error, no range and no candidates. Review finding #8. * ci(e2e): name a failed API read in the red-run report and derive the leg target text The report runs under set -e, so a failed gh api read before the annotation left a red run with no code. An ERR trap now writes E2E_REPORT_FAILED with the next action. The over-target summary and warning print the target from TARGET_SECONDS instead of a fixed 10m. * docs(e2e): note that a bot release push does not run the e2e legs * ci(checker-parity): drop the run step's cap, which equals the job's and can never fire first
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
Fixes full-path receipt findings #8 (four-channel-contracts) and #11 (tagged-unions-over-state-by-presence) on a359a82. The settle command took restoreError and runFailure as nullable Cause.pretty strings and decided on their presence. It now takes two tagged unions, RunPathMoveBack (MovedBack | MoveBackFailed { cause }) and RunPathRunEnd (RunSucceeded | RunFailed { cause }), and matches on the move-back. RunPathUnrestored carries cause: S.optional(S.Unknown) and the run's end instead of strings; the service passes each Exit's Cause as is. Text is rendered once, in the error's message, when the report's reason is built. The settle law draws the production SettleRunPathCommand and asserts unrestored iff the move-back failed, with both causes carried by identity. Plan: docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U7.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
…s-through
resolveImportedExport only projected resolveCallee's Callee down to { module, exportName } for its single caller. Call resolveCallee directly in Transformer.service and build the arid EffectExport callee inline; delete resolveImportedExport and the ResolvedEffectExport interface.
Finding #8 (review run 20261013-025925-88221c72).
ryanleecode
added a commit
that referenced
this pull request
Oct 13, 2026
…mports (#290) * feat(stryker-js-instrumenter)!: resolve arid callees through effect imports Plan U6 (R8, R9; KTD6, KTD7). The arid callee fact is now a resolved EffectExport { module, exportName } from the file's effect imports, or an unshadowed console/Date Global, so an aliased namespace or a bare named import is arid and a local look-alike is not. Effect.fn('name', options) makes its name and options arid under arid-telemetry; the body passed to Effect.fn stays mutated. EffectCall's import table is parameterized by module list; resolveEffectCall keeps its closed set. * refactor(stryker-js-instrumenter): resolve arid callees only for mutated argument frames ce-simplify-code on U6: the arid reason is computed only for nodes with replacements, and the callee only when the mutant sits in a call argument; global names short-circuit before the shadow scan. AridEffectModule and AridGlobalObject are closed literal schemas typing the callee union. Match.tagsExhaustive replaces hand-rolled tag dispatch; the import table carries its module list; Transformer reuses Mutator's isStringLiteral. * refactor(stryker-js-instrumenter)!: move effect import resolution into effect-imports and stop arid at function boundaries Apply the S1 L3 review rulings (it_bdfe56a0): the import table, resolveImportedExport and isImportedLocal move into one pure module, effect-imports.ts, shared by EffectCall and the Transformer; the arid table is built once per Program. A FunctionBoundary frame ends the arid walk, so a mutant inside a function passed to an arid callee is mutated. Effect.fn non-function arguments are arid by position. arid-time drops Date.now. Property tests derive generators from AridCalleeSchema and state the plan's no-effect-module law. * test(stryker-js-instrumenter): pin arid suppression so the property file refutes a constant aridCode CI's VacuousProperty gate found that every arid law held against a constant impostor. Any Logger export or console member drawn from AridCalleeSchema must be suppressed as arid-logging under its canonical name. * docs(plans): supersede the mutant-quality plan with a v2 that states the shipped Effect.fn rule Review finding 1 on #290: R9, KTD7 and U6's Approach still prescribed the deleted string-first guard. Implementation had started (L1 merged as #262), so the correction is a new dated plan declaring supersedes, not an in-place edit. v2 also reads mutation evidence only from nightly or manually dispatched main Mutation runs (2026-10-12 mutation law). * docs(stryker-js-plugin-interface): say a call inside a function still covers its mutants Review finding 4 on #290: the arid rule text said a mutant inside a function is never covered, but a call frame inside the same function still covers it (the walk stops only at the function boundary). Changeset mirrors the wording. * refactor(stryker-js-instrumenter): model the arid call frame as a tagged union AridCallFrame carried a callee Option correlated with the childIsArgument boolean, and both producer and consumer guarded that correlation by hand. Split it into an ArgumentFrame (carries the callee Option, which the producer can leave unresolved) and a ThroughFrame (no callee), dispatch with Match.tagsExhaustive, and re-express the correlation-pinning law as 'a through frame never suppresses' so no coverage is lost. Finding #6 (review run 20261013-025925-88221c72). * refactor(stryker-js-instrumenter): drop the resolveImportedExport pass-through resolveImportedExport only projected resolveCallee's Callee down to { module, exportName } for its single caller. Call resolveCallee directly in Transformer.service and build the arid EffectExport callee inline; delete resolveImportedExport and the ResolvedEffectExport interface. Finding #8 (review run 20261013-025925-88221c72). * docs(stryker-js-instrumenter): restore the EffectCall module contract header The arid-imports refactor deleted the EffectCall.ts contract block instead of moving it. Reword it into effect-imports.ts as the header for what that module owns — import table, callee resolution, shadow analysis, built once per Program and passed down — and give EffectCall.ts a one-paragraph header naming what stays (call form, arity, module access for replacements) and pointing to effect-imports.ts. Finding #7 (review run 20261013-025925-88221c72). * test(stryker-js-instrumenter): drop the vacuous Date refusal scenarios The two Date scenarios asserted Date.parse('x') mutants stay mutable, but the head restricts the arid globals to 'console' and removed the Date.now rule, so no rule could ever make those mutants ignored. They cannot fail when their guard is removed; the imported-console and shadowed-console scenarios already falsify a dropped isImportedLocal/isShadowed. Delete both scenarios and their ARID_IMPORTED_DATE_SOURCE/ARID_SHADOWED_DATE_SOURCE constants. Finding #1 (review run 20261013-025925-88221c72). * fix(stryker-js-instrumenter): satisfy oxlint for the arid tagged frames Dispatch the call-frame predicate with Match.tagsExhaustive instead of touching _tag directly (no-direct-tag-access), and extract aridCallFrame so aridCallFrameFor stays within the complexity cap. Follow-up to finding #6 (review run 20261013-025925-88221c72). * docs(plans): supersede the mutant-quality plan v2 with v3 naming effect-imports.ts and the removed Date rule L3 review findings 2 and 5 (run 20261013-025925-88221c72): KTD6 still named EffectCall.ts as the import table's home and Date as an arid global; U6 Files, Patterns and U13 citations pointed at EffectCall.ts. v2 was in execution, so the correction is a new dated plan declaring supersedes. --------- Co-authored-by: Ryan Lee <drdgvhbh@gmail.com>
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
…s decision table Findings #2, #7 and #8 (review run 20261013-032451-a9153d08, validator-confirmed): the pure collection-size decision shipped without the property file the package plan requires beside every workflow. Add classify-collection-size.workflow .property.test.ts, a schema-driven law that draws ClassifyCollectionSizeCommand values from the production Schema and pins the workflow's verdicts to an independently written decision table (both length sides, every operator including the mirrored right-side cells, every guard: against zero, reference chain, strictNullChecks, receiver kinds and the kept constant sibling).
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
… run The unreadable-plan row asserted only the reason text and the exit class, unlike its sibling row for an undecodable plan. Assert the plan-unreadable code and the next: check the --plan path guidance the CLI prints. Finding #8 of review run 20261013-032449-0d3f2153.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
…s suffix Fixes c0cf821b #8 (cell-architecture service-and-layer-boundaries, confirmed): command.service.ts and plan-legs.service.ts were driver-backed modules (ChildProcessSpawner, gh, git, turbo) under the suffix the rule reserves for a Context.Service contract, and test/bench declares no Context.Service. They become command.ts and plan-legs.ts, names the rule leaves free for a driver-backed module, and their three importers follow. prepare-side.service.ts and run-bench.service.ts predate this PR and are unchanged (the review ruled them pre-existing). Plan: docs/plans/2026-10-13-0658-ci-bench-off-pr-plan.md U24.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
…he vacuity gate refuses CI run 38363051350 (check, job 115182283579) failed the suite: the vacuity gate found that a constant [] impostor held for 100 runs of ∀command_Decisions_≡IndependentDecisionTable, because Arbitrary.schema(ClassifyCollectionSizeCommand) rarely draws a kept, qualifying comparison beside its kept literal. The law also read its answers from a hand-written known-answer table. The architect-property-tests rule deletes a law a gate refuses rather than reshaping it, and keeps a law only when no other suite pins the behavior. check-mutants.integration.test.ts drives the built checker over every constant row (xs.length >= 0, xs.length < 0, 0 <= s.length, 0 > s.length) and every kept guard (non-zero bound, no backing sibling, non-constant operator, non-collection and mixed receivers, call, element, optional and await receivers, accessor, tuple, this, strictNullChecks off). Ruling S1 F1 deleted this file's earlier vacuous law on the same grounds. Supersedes the law added for findings #2/#7/#8 of review run 20261013-032451-a9153d08 (fe6b1e5) and reshaped for #10/#11 of run 20261013-055603-242b8fc9 (28da5be).
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
`projects` is `command.projects.length`, a non-negative count, which the package idiom types as `S.Natural` (`cap-survivors.workflow.ts:19`), not `S.Int`. Rule schema-laws/invariants-as-refinements; full-path ce-code-review run 20261013-061137-3bcdcd0e finding #8.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
Rename the started plan to 2026-10-13-0803 (supersedes the 0732 file, which it replaces) and record revision 3, clearing the three schema-laws findings of full-path ce-code-review run 20261013-061137-3bcdcd0e: A requires `thresholds` in GateReportDocument (tagged-unions-over-state-by-presence, #1), B states `percentage < threshold` on BreakBreach and builds one only for a real breach (cross-field-invariants-as-struct-checks, #6), C counts projects with `S.Natural` (invariants-as-refinements, #8). The changeset notes that a report with no `thresholds` is now refused.
ryanleecode
pushed a commit
that referenced
this pull request
Oct 13, 2026
* ci(bench)!: bench each main commit against its first parent
The bench leaves the pull-request path. bench.yml runs on push to main
(docs-only pushes excluded) and on workflow_dispatch, with concurrency
bench-${{ github.sha }} and no cancel, so no PR pays for or waits on it.
- Side A is the first parent, side B the pushed commit; both are worktrees
under RUNNER_TEMP with their own released tarballs and frozen install,
and the checkout holds only the harness.
- Every in-process deadline is gone: BENCH_DEADLINE_MS, the setup budget,
the run deadline, BENCH_RUN_TIMEOUT_MS, and the setup-timed-out,
budget-exceeded, timed-out and timeout shapes. Every timeout-minutes is
60; that job timeout is the only kill.
- The report carries the leg's wall time (from JOB_STARTED_AT) against a
10-minute target, never enforced, and schemaVersion moves to 2.0.
- Only a leg's total decides a regression (non-overlapping samples and
|delta| >= max(3% of A, 10 ms)). The regression block names the commit,
its parent, total then every regressed phase, and the next action: a
revert PR or root-fix P0 on main, speed evidence only elsewhere.
- Every run executes from RUNNER_TEMP/bench-run, so both sides record the
same project root, inputs digest and mutant ids. Bench configs reference
plugins relative to the config file so they survive the move.
Plan: docs/plans/2026-10-12-1800-ci-bench-off-pr-plan.md (layer 1; the
enterprise install is layer 2).
BREAKING CHANGE: bench-report.json schemaVersion 2.0 drops the
setup-timed-out and budget-exceeded abort codes and the timed-out run exit,
and adds ref, leg and the regression block.
* docs(bench): say an environment-incomplete abort writes no report
Review finding (correctness, P3): the README said every aborted leg uploads
its report, but main.ts publishes environment-incomplete without writing
bench-report.json or the step summary.
* ci(bench): drop the hand-written path filter from the main trigger
Root amendment 2026-10-12: no change classifier where Turbo can do the job.
paths-ignore (**/*.md, docs/**) skipped none of 11 sampled main commits;
turbo query affected skips three of them. Every push now benches every
leg until layer 1b scopes legs with turbo query affected.
Plan: docs/plans/2026-10-12-1854-ci-bench-off-pr-plan.md supersedes the
1800 plan and records the turbo-query evidence and rule-4 reasons.
* docs(plan): bench every main SHA; defer Turbo affected-only leg selection
Sub-conductor ruling 2026-10-12: the acceptance is a comparison per main
SHA, so Turbo's affected-packages answer stays evidence, not a leg filter.
The 1926 plan supersedes 1854: layer 1b is dropped, KD10 keeps Turbo as
the owner of builds and cache and adds no classifier, KD11 defers
affected-only selection to root.
* fix(bench): name phase-only slowdowns instead of claiming no phase changed
A summarized report with no regression said "no project/phase changed past
the signal threshold" even when its table showed a phase that regressed under
an unchanged total. The notice is now titled "Bench phase-only slowdown", says
no leg total crossed the regression threshold, and lists each such cell as a
phase-only slowdown. A new law drives a check-only slowdown through
summarizeBench and renderBenchReport.
The plan moves to docs/plans/2026-10-12-2032-ci-bench-off-pr-plan.md
(supersedes 1926) and adds the CONST-W3 declaration for the bench.yml edits,
the corrected cause of the stream-undecodable PR runs (#298 changes the
stream contract), and that case as a known limit.
* docs(plan): list every bench.yml edit in the CONST-W3 waiver and scope the PR-run evidence
Review of eec5cfa found that the waiver called side B the checkout (it is the bench-b worktree) and left out the checkout ref, Turbo cache and harness install edits. Run 38284821646 also had a budget-exceeded abort on one leg, so the Problem Frame now scopes stream-undecodable to the other three legs.
* test(e2e-core): move the bench decision checks beside their workflows as laws
bench-decisions.integration.test.ts fed hand-written NDJSON lines and
report text to the pure readBenchRun and coveringTestFiles workflows and
asserted their decision fields. Under root's 2026-10-12 testing law an
integration test drives the real product and asserts only observable
outcomes, and a pure decision gets property laws beside it (CONST-T12,
CONST-T14). This layer had to change the file (ReadBenchRunCommand's
exit field), so it is deleted rather than carried.
The behaviours no other suite pinned are now laws over the production
schemas: testsExecuted sums every drawn MutantCost, the first undecodable
line is named by its 1-based number among blank and decodable lines, a
verdict count other than one is stream-invalid, verdict phase durations
are carried as reported or refused when absent, covering test files are
the sorted distinct holders, and a non-JSON report is refused. The
legacy-line decode stays pinned by the cli-contract wire-line suite.
The plan moves to docs/plans/2026-10-12-2154-ci-bench-off-pr-plan.md
(supersedes 2032) with the widened edit scope and test admission rows.
* refactor(bench): keep the bench's new decisions in test/bench
Root's 2026-10-13 scope law puts no new code in test/e2e-core. The
layer's four new modules and their laws move to test/bench/src, which is
their only consumer: run-path.schema, plan-run-path and settle-run-path
(the fixed run path) and measure-leg (the leg wall time). e2e-core's
mod.ts is back to main.
measureLeg now decides a bench-local LegClock (LegElapsed or
LegClockBehind) that main.ts maps onto the report's LegTime:
make-body-purity forbids a decision from constructing a class imported
from another package, and e2e-core cannot import test/bench.
test/bench gains a test lane shaped like test/checker-parity: a test
script, a vitest config with inSourceSchemaLaws, tsconfig.test.json and
lint:tsgo over it. The plan moves to
docs/plans/2026-10-13-0223-ci-bench-off-pr-plan.md (supersedes 2154).
* fix(bench): abort an outside-tree run path as a defect
Review of f1da499 (MiniMax lenses, report-only) found two defects in
the bench's run path:
- planRunPath's identity law drew cwd and CLI tails of length one or
more, so the enterprise shape (cwd equal to the tree) was never
generated. Dropping the `path === tree` operand left the law green;
with empty tails allowed it now fails.
- A cwd or CLI outside its side's tree aborted as run-path-move-failed,
whose next action says to fix the runner. It is a bench bug, so it
now aborts as defect. The README lists that case.
The plan moves to docs/plans/2026-10-13-0238-ci-bench-off-pr-plan.md
(supersedes 0223) with an accurate split of the e2e-core law files.
* fix(bench): name both causes in the run-path-move-failed action
63ba7bb sent an outside-tree run path to `defect`. Review of that head
found two problems: it contradicted KTD3, and `defect`'s next action asks
for a stack trace, which a typed failure does not have. So the outside-tree
case aborts as run-path-move-failed again. That code's next action, which
this layer added, now names both causes. An OS error needs a runner fix; a
path outside the tree is a bug in side setup. The README matches.
The plan moves to docs/plans/2026-10-13-0258-ci-bench-off-pr-plan.md
(supersedes 0238), with U4 brought back in line with KTD3.
* docs(plan): plan the pack-violation fixes for the bench lane
Supersedes the 0258 plan. Root (2026-10-13) requires every confirmed pack
violation in the a359a82 full-path receipt to be fixed, not waived.
Adds KD12-KD15 and units U7-U12, the e2e-core boundary (one in-source
refusal block per refined e2e-core file, LegTime as a data union), and
test-admission rows for the refusal properties and the settle law.
* fix(bench): carry run-path settle outcomes as tagged causes
Fixes full-path receipt findings #8 (four-channel-contracts) and #11
(tagged-unions-over-state-by-presence) on a359a82.
The settle command took restoreError and runFailure as nullable
Cause.pretty strings and decided on their presence. It now takes two
tagged unions, RunPathMoveBack (MovedBack | MoveBackFailed { cause }) and
RunPathRunEnd (RunSucceeded | RunFailed { cause }), and matches on the
move-back. RunPathUnrestored carries cause: S.optional(S.Unknown) and the
run's end instead of strings; the service passes each Exit's Cause as is.
Text is rendered once, in the error's message, when the report's reason
is built.
The settle law draws the production SettleRunPathCommand and asserts
unrestored iff the move-back failed, with both causes carried by
identity. Plan: docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U7.
* fix(bench): make the report's leg time plain data
Fixes full-path receipt finding #9 (data-only-schema-classes) on a359a82.
LegMeasured and LegNotMeasured were TaggedClass variants carrying a
LegTime type id that no Workflow.make checks; LegTime is only a field of
BenchReport. LegTime is now an S.TaggedUnion of the same two cases, so
bench-report.schema.ts declares two fewer classes and no type id, and the
encoded shape is unchanged. main.ts and the render law construct through
LegTime.cases.
Evidence: tsc, oxlint and lint:tsgo clean in test/e2e-core and
test/bench; vitest 364/364 in test/e2e-core (the 3 fewer tests are the
generated codec laws of the two deleted classes; LegTime's own laws
remain) and 39/39 in test/bench. Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U8.
* fix(bench): refine the leg's elapsed time and the report's ref
Fixes full-path receipt finding #10 (invariants-as-refinements) on
a359a82, both fields named by root.
- LegElapsed.wallMs was S.Finite, so a negative elapsed time decoded as
the "clock advanced" variant. It is now finite and >= 0, with the
invariant stated in the check's message.
- BenchReport.ref was S.String while SummarizeBenchCommand.ref is
S.NonEmptyString. It is now S.NonEmptyString, and main.ts reads
BENCH_REF with Config.NonEmptyString, so an empty value takes the
existing environment-incomplete path (main.ts program, UNKNOWN_REF)
instead of failing at BenchReport.make.
Evidence: tsc, oxlint and lint:tsgo clean in test/e2e-core and
test/bench; vitest 364/364 and 39/39. The refusal properties for these
refinements land with #12. Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U9.
* fix(bench): state the refusal boundary of every refinement this PR adds
Fixes full-path receipt finding #12 (refusals-beside-generated-laws) on
a359a82.
The generated schema laws draw only accepted values. Each refined schema
the PR adds or changes now states its refusals in its own file, in one
import.meta.vitest block whose subject is a private predicate. Each
block draws the unrefined base, appends boundary values to every draw,
and compares acceptance with a predicate written from the domain:
- test/bench: run-path.schema.ts (all ten path fields), leg-clock.schema.ts
(elapsed ms finite and >= 0), and the commands of plan-run-path,
settle-run-path and measure-leg.
- test/e2e-core, the narrowest arrangement under root's scope law (plan
KD13: no new file, export, dependency, config or helper):
bench-report.schema.ts (LegTargetMs, ref, abort codes = the codes with a
next action, rendered exit 0|1), bench-summary.schema.ts (RegressedMetric
and PhaseRegression fields), bench-run.schema.ts (failure codes without
timeout, exitCode a safe integer, never null) and summarize-bench (ref).
Evidence: tsc, oxlint and lint:tsgo clean in both packages; vitest
44/44 in test/bench (+5) and 375/375 in test/e2e-core (+11). The pack's
widen-once check is not run: root forbids hand-applied probes. Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U10.
* ci(bench): give each dispatch its own concurrency group
Fixes full-path receipt finding #15 (heavy-suites-judge-every-main-commit)
on a359a82.
The group was bench-${{ github.sha }} for both triggers, so a
workflow_dispatch on main's head shared, and could replace, the pending
push run for that commit. A push now groups by github.sha and a
dispatch by github.run_id, with cancel-in-progress false, as the rule's
Concurrency bullet requires. Why side A is HEAD^1 rather than the last
green commit is stated in the lane's cost header (#14). Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U11.
* ci(bench): state the lane's measured runner-minutes in its header
Fixes full-path receipt finding #14 (heavy-lanes-state-their-daily-minutes)
on a359a82.
bench.yml now opens with the cost header the rule asks for:
- the daily figure, about 267 runner-min/day (241-319), priced on the
run every push to main starts;
- that run's shape, the full corpus against the first parent, and why
side A is HEAD^1 rather than the last green commit (the corpus is a
fixed workload; an older base selects no smaller workload and would
credit several pushes' change to one commit), which also answers #15;
- the per-run cost, 15.3-20.3 job-min, with the seven run ids;
- the 7-day first-parent merge count, 110 in 2026-10-06T01:54Z to
2026-10-13T01:54Z (15.71/day);
- whether the lane can starve PR CI, from the same week's job
timestamps: only once about 40 jobs are already running; the replay
without the pull_request Bench lowers hours at 40+ jobs from 11.98 to
11.45.
Every figure is from GitHub Actions job timestamps already reported in
the PR's Bench cost section; nothing was rerun. Plan:
docs/plans/2026-10-13-0416-ci-bench-off-pr-plan.md U12.
* docs(plan): record the landed concurrency, settle law and header shape
Supersedes the 0416 plan. R3, U1 and the CONST-W3 list now carry the
push-sha/dispatch-run-id concurrency group from 7d962fc. U7's test
sentence matches the landed settle law (it draws the production command
and asserts the carried causes, not message text, whose oracle would
copy Cause.pretty). U12 records that the HEAD^1 rationale lives in the
cost header.
* docs(plan): say where the HEAD^1 rationale lives in U11
Supersedes the 0434 plan. U11 no longer claims an inline comment; the rationale is in the cost header (U12).
* docs(plan): plan the fixes for every confirmed row of the b30908c receipt
Supersedes the 0435 plan. Root's merge bar (2026-10-13) requires a fixing
commit on the reported head for every confirmed pack violation and accepted
finding. Adds KD16 (push runs scoped from the last green main commit, legs
selected by Turbo's affected set), KD17-KD19 and units U13-U18.
* fix(bench): build the report-unwritable fallback with the report builder
Fixes b30908c full-path receipt finding #5 (maintainability: the fallback
hand-rebuilt every BenchReport field). reportOf now takes report runs, and
writeReport's fallback calls it with the report's own fields.
Plan: docs/plans/2026-10-13-0554-ci-bench-off-pr-plan.md U13.
* fix(bench): walk the phase verdicts once in the report renderer
Fixes b30908c full-path receipt finding #4 (maintainability: two
near-identical verdict walks). One traversal yields each improved or slower
phase change; the improved and phase-only lists filter it. Only lines this
PR added change.
Plan: docs/plans/2026-10-13-0554-ci-bench-off-pr-plan.md U14.
* fix(bench): move the corpus schema into test/bench and state its refusals
Fixes b30908c full-path receipt finding #2 (testing: the corpus empty-repo
refusal lost its only test when bench-decisions.integration.test.ts was
deleted).
bench-corpus.schema.ts is consumed only by the bench, so it moves from
test/e2e-core/src to test/bench/src, unchanged, and e2e-core's mod.ts drops
its export. Beside it, one in-source refusal block states every refinement
the corpus carries: each text field refuses only the empty string, and each
list (repo, mutate, testFiles) refuses only the empty list. The repo list's
refusal is the empty-repo case the deleted test covered; the bench aborts
such a corpus as corpus-unreadable.
test/bench: 54 tests (44 + 2 refusal properties + 8 generated codec laws);
test/e2e-core: 367 (375 less the moved schema's 8 codec laws).
Plan: docs/plans/2026-10-13-0554-ci-bench-off-pr-plan.md U15 (amended in
the next plan revision: a move, not a workflow).
* fix(bench): decide every run-path abort in a lawed workflow
Fixes b30908c full-path receipt finding #1 (testing: the three run-path
abort codes were produced only in run-bench.service.ts, which no test
reaches).
Each way the run path can fail is now a tagged failure carrying its cause:
RunPathUnlisted, RunPathOutsideTree, RunPathOccupied, RunPathMoveInFailed
and RunPathUnrestored (run-path.schema.ts). abortRunPath
(abort-run-path.workflow.ts) decides the abort for each: occupied ->
run-path-occupied, unrestored -> run-path-unrestored, the rest ->
run-path-move-failed, with a reason that names the side and the paths.
run-bench.service.ts keeps only the listing, the two renames and the run,
and routes every failure through the decision.
The law draws RunPathFailure from its production schema and checks the code
and that the reason names the side and paths. In-source refusal properties
cover the new path fields and each abort variant's code. An in-process test
of the renames is refused by the gates (tests/ may import only a package's
public surface, and the bench has none), so the A/A dispatch remains the
evidence for the rename order. test/bench: 68 tests.
Plan: docs/plans/2026-10-13-0554-ci-bench-off-pr-plan.md U16 (amended in
the next plan revision: a lawed decision, not an integration test).
* refactor(bench): call side A the base, not the parent
Side A is no longer always the first parent: on a push it is the lane's
last green main commit. SummarizeBenchCommand.parentSha and
PhaseRegression.parentSha become baseSha, and the regression annotation,
the step-summary table header and the regression section say "base".
Every renamed line is one this PR added (origin/main has no parentSha or
"parent" wording in these files), so the e2e-core boundary holds.
Plan: docs/plans/2026-10-13-0554-ci-bench-off-pr-plan.md KD18.
* fix(bench): bench a push from the lane's last green main commit, affected legs only
Fixes b30908c full-path receipt finding #7 (ci-wall-clock
heavy-suites-judge-every-main-commit: bench.yml compared each push with
HEAD^1 instead of the lane's last green commit).
A plan job now picks side A and the legs before any bench job starts:
- selectLegs (select-legs.workflow.ts) is the decision. On a push, side A is
the newest main commit whose push-triggered Bench run concluded success and
which is an ancestor of the pushed commit. A leg runs when Turbo's affected
set holds the package containing its entry or an engine directory every leg
builds. The full corpus runs on a dispatch, when no green ancestor exists
(side A is then the first parent), and when the lane's own inputs changed
(test/bench, bench.yml, the released-tarballs action, the flake pin). A
commit with no green ancestor and no first parent is refused as NoBenchBase.
- plan-legs.service.ts gathers the inputs: the lane's green push runs (gh api),
ancestry and the diff (git), and `turbo query affected --packages`. It fails
when the checkout has uncommitted changes, because Turbo adds them to the
affected set (seen locally: a dirty test/bench marked every leg affected).
- plan.ts writes `base` and `legs` to GITHUB_OUTPUT and the choice to the step
summary; bench.yml's plan job runs it with `actions: read`, the bench job
uses its base for side A and BENCH_BASE_SHA, and it is skipped when no leg
is selected.
- engine-dirs.ts holds the engine directories, now shared by prepare-side and
the plan; command.service.ts holds the command runner, moved out of
prepare-side so the plan reuses it.
Laws: select-legs covers dispatch, no green ancestor, lane inputs changed, and
affected-only selection (drawing a prefix-without-slash path; the law refutes a
`startsWith` without the `/` boundary). In-source refusal laws cover the new
schemas. Smoke: the real plan program on this checkout (push, dispatch) and,
with a gh shim naming the parent as green, on 3cb1342, ddeb67a and 61d40db.
Plan: docs/plans/2026-10-13-0554-ci-bench-off-pr-plan.md U17, KD16.
* docs(plan): record how U15-U17 landed
Supersedes the 0554 plan. U15 moves the corpus schema into test/bench with
its refusals instead of adding a resolveTarget workflow; U16 decides every
run-path abort in a lawed workflow, because the gates refuse an integration
test of a package with no public surface; U17 adds plan-legs.service.ts
(refusing a dirty checkout, since Turbo counts uncommitted changes) and moves
the command runner into command.service.ts. The edit scope records the one
file that leaves e2e-core, and the test admission table records the refused
and withdrawn tests.
* ci(bench): price the cost header for affected-only push runs
Keeps b30908c receipt row #14 (heavy-lanes-state-their-daily-minutes)
true after #7's selection changed the run shape.
The header now states the plan-job-then-selected-legs shape, each leg's
median job-minutes (range) over the seven dispatch runs, the plan job's
0.40-0.71 min (the same setup steps as timed in those runs, plus the plan
step timed locally), and the week's price: the real plan program run over
all 110 first-parent main commits of 2026-10-06T01:54Z..2026-10-13T01:54Z,
each with its parent as side A, picks no leg for 16, one for 2 and four for
92 (64 for changed lane inputs), so 1,655 job-min (1,333-1,948), about 236
runner-min/day (190-278). The PR CI replay stays as measured for the
full-corpus shape; the selection only removes jobs from it.
Plan: docs/plans/2026-10-13-0629-ci-bench-off-pr-plan.md U18.
* docs(plan): plan the fixes for the c0cf821b review's confirmed findings
Supersedes the 0629 plan. Fixes c0cf821b #4 (KTD2 named a
run-path.service.ts that never landed) and the plan half of #6 (R13 and the
summary still said first-parent side A).
KD20 records root's merge bar for this round: every confirmed finding of the
c0cf821b review of 5a088c6 gets a fixing commit, and the final receipt is a
new full-path review of the head that holds them. U19-U26 plan one fix per
confirmed finding group: the package description, an escaped plan-job
annotation, the plan job's wall time in its step summary, Turbo's wire decoded
into paths, driver modules off the reserved *.service.ts suffix, a lawed
green-ancestor choice, and the run's regression as a tagged union.
* docs(bench): describe the green-ancestor side A in the package description
Fixes c0cf821b #1, #3 and #6 (correctness, project-standards and
maintainability, all confirmed): test/bench/package.json still said each
push is benched against its first parent. It now states the rule the plan
job applies, the same one README.md and the bench.yml header state.
Plan: docs/plans/2026-10-13-0658-ci-bench-off-pr-plan.md U20.
* fix(bench): escape the plan job's ::error annotation
Fixes c0cf821b #5 (maintainability, confirmed): failWith printed
`::error title=<title>::<message>` raw, and a refusal's message can hold
multi-line command output (git status, gh's error body), which breaks the
single-line workflow command. The title is now escaped as a property (%,
CR, LF, colon, comma) and the message as data (%, CR, LF), the same
replacement set e2e-core's renderer uses; that helper is private and KD13
forbids a new e2e-core export, so plan.ts keeps its own copy.
Smoke: a plan run against a repository gh cannot find prints one
`::error title=Bench plan failed::...%0A...` line.
Plan: docs/plans/2026-10-13-0658-ci-bench-off-pr-plan.md U21.
* fix(bench): write the plan job's wall time to its step summary
Fixes c0cf821b #13 (ci-wall-clock one-stuck-job-backstop, confirmed): the
plan job had the 60-minute backstop but wrote no wall time to its step
summary, so a stuck plan job could not be told from a slow one.
The plan job's first step now records JOB_STARTED_AT, as the bench job's
does, and plan.ts appends the job's wall time in a finalizer, so the line is
written after a selection, a refusal and an incomplete environment alike.
Smoke: the plan program on a dispatch (exit 0), on an unknown repository
(exit 1) and with no GITHUB_SHA (exit 1) each appended
"**Plan job wall time:** N s through the plan step".
Plan: docs/plans/2026-10-13-0658-ci-bench-off-pr-plan.md U22.
* fix(bench): decode Turbo's affected-packages JSON into the paths the plan uses
Fixes c0cf821b #10 (schema-laws data-only-schema-classes) and #11
(schema-laws rich-type-over-foreign-encoded), both confirmed:
TurboAffectedPackage and TurboAffected were schema classes no contract
required, and plan-legs walked Turbo's data.affectedPackages.items envelope
itself.
The wire is now a private S.Struct, and TurboAffectedPaths decodes it with
S.decodeTo into ReadonlyArray<string>, so plan-legs receives the paths and
nothing of Turbo's envelope. The generated codec laws now cover
TurboAffectedPaths in place of the two classes.
Plan: docs/plans/2026-10-13-0658-ci-bench-off-pr-plan.md U23.
* refactor(bench): move the driver modules off the reserved *.service.ts suffix
Fixes c0cf821b #8 (cell-architecture service-and-layer-boundaries,
confirmed): command.service.ts and plan-legs.service.ts were driver-backed
modules (ChildProcessSpawner, gh, git, turbo) under the suffix the rule
reserves for a Context.Service contract, and test/bench declares no
Context.Service. They become command.ts and plan-legs.ts, names the rule
leaves free for a driver-backed module, and their three importers follow.
prepare-side.service.ts and run-bench.service.ts predate this PR and are
unchanged (the review ruled them pre-existing).
Plan: docs/plans/2026-10-13-0658-ci-bench-off-pr-plan.md U24.
* fix(bench): choose side A in a lawed decision, the newest green ancestor
Fixes c0cf821b #2 (testing, confirmed): the push-path side A choice lived
in plan-legs, which took the first SHA in the API's listing that git
called a strict ancestor. No test covered it, and the choice depended on
the API's list order.
chooseBase (new workflow) takes every green Bench push run on main
(GreenRun: SHA, creation time in epoch seconds, and whether it is an
ancestor of the pushed commit), the event and the first parent. On a
push it picks the newest run that is an ancestor and not the pushed
commit, else the first parent ('no-green-ancestor'); on a dispatch, the
first parent ('dispatch'); with no first parent, NoBenchBase. selectLegs
now takes that BaseChoice plus the diff against it and can no longer
fail. The base must be chosen before the diff is computed, so the choice
cannot live inside selectLegs as U25 first said; the plan amendment
follows.
plan-legs lists the runs through gh's --jq (created_at via
fromdateiso8601), decodes them with GreenRunsJson, asks git once per run
whether it is an ancestor, and computes the diff only for a GreenBase.
Laws: chooseBase's dispatch and push rules over drawn GreenRuns, with the
head drawn as a green run's SHA. Two mutants were refuted: taking the
first listed run, and not excluding the pushed commit. selectLegs' laws
are now keyed on the base they receive. Refusal laws cover GreenRun's
safe-integer seconds, FirstParentReason and both commands.
Plan: docs/plans/2026-10-13-0658-ci-bench-off-pr-plan.md U25.
* docs(plan): choose side A in its own decision before selecting legs
Supersedes the 0658 plan. U25 as written put the green-ancestor choice
inside selectLegs, but selectLegs reads the diff against side A, so the
base has to be chosen before that diff exists. U25 now names chooseBase
(new) as the decision; selectLegs takes its BaseChoice. The Test Admission
table splits the select-legs row, adds the choose-base law file with the
bugs it catches, and records the U25 smoke for plan-legs.
* fix(e2e-core): decide a run's regression as a tagged union, not NullOr
Fixes c0cf821b #12 (schema-laws tagged-unions-over-state-by-presence,
confirmed): BenchSummary.regression and the summarized report outcome were
S.NullOr(PhaseRegression), so "no leg regressed" was the absence of a
value, and the renderer probed it with Option.fromNullishOr.
Regression = S.TaggedUnion({ PhaseRegressed, NoLegRegressed }) replaces
PhaseRegression and its 'phase-regressed' code literal. summarizeBench
decides NoLegRegressed explicitly, and renderBenchReport matches the two
tags with Match.valueTags for both the annotation and the markdown block.
The annotation and block title stay 'phase-regressed', now a renderer
constant, so the rendered verdict line is unchanged. The refusal law on
the code literal becomes one on the discriminant: only the two tags
decode.
The law files lose their `=== null` / `!== null` checks, which still
compiled against the union but would have passed vacuously; each is now a
tag check. test/bench/README.md says the JSON report carries the tag.
origin/main has none of these lines, so every edited line is one this PR
added (KD13). report schema 2.0, bumped by this PR, covers the shape change.
Plan: docs/plans/2026-10-13-0711-ci-bench-off-pr-plan.md U26.
* docs(plan): plan the b2fe30db review's fixes and route two findings to root
Supersedes the 0711 plan. Fixes b2fe30db #2 (testing-confirmed: KD13 said
no e2e-core export was added, but the pack fixes added LegTargetMs,
LegTime, RegressedMetric, Regression and PhaseRegressed) and answers #1
(feb5085's subject is scoped e2e-core; rewriting it needs a force push,
and the squash merge carries only the PR title).
KD21 records the round. U27-U29 fix #12 (GreenRunsJson loses its
generated laws), #11 (corpus schemas as data classes) and #6 (two
stderr-tail helpers). #9 (pin Turbo's --packages output) and #3 (test
atRunPath's renames) ask for tests the admission gate refuses: a Turbo
pin spawns git and turbo, and atRunPath is a private executor helper. The
repository's real-Turbo pins live in scripts/ under PR #319, so both go
to root. The Turbo semantics the plan job relies on were measured on
2.10.12 in a scratch repository and are recorded in KD21.
* fix(bench): root the green-run listing codec at the schema vocabulary
Fixes b2fe30db #12 (schema-laws rich-type-over-foreign-encoded,
confirmed): GreenRunsJson was GreenRunListing.pipe(...), a chain rooted
at a module-local name, which effect-schema-discovery skips. The codec
the plan job decodes gh's output with therefore had no generated
round-trip or encode-stability law. It is now
S.Array(GreenRunListing).pipe(S.fromJsonString), and
∀x_GreenRunsJson_=x and ∀x_GreenRunsJsonEnc_=x run.
Plan: docs/plans/2026-10-13-0749-ci-bench-off-pr-plan.md U27.
* fix(bench): declare the bench corpus as plain data
Fixes b2fe30db #11 (schema-laws data-only-schema-classes, confirmed):
BenchRepoEntry, BenchEnterpriseCorpus and BenchCorpus were S.Class
declarations, though none is a Workflow.make command, a decision variant
or a tagged error; every consumer only reads their fields. They are now
S.Struct schemas with exported types. BenchCorpusJson keeps its S.*-rooted
chain, so its generated laws still run, and the refusal block is
unchanged.
Plan: docs/plans/2026-10-13-0749-ci-bench-off-pr-plan.md U28.
* refactor(bench): keep one stderr-tail helper for the bench's commands
Fixes b2fe30db #6 (maintainability, confirmed): command.ts kept a
4000-character tail over a CommandOutcome, and run-bench.service.ts a
second 4096-character tail over a string, both added by this PR.
command.ts now owns the one budget (4096, private) and exports
tailOf(text); runChecked, the interrupt message and runCli's stderr
collector all use it. A setup failure keeps 96 more characters of
output.
Smoke: a plan run whose gh exits 7 after 6000 bytes of stderr fails with
an annotation that carries the last 4096 characters.
Plan: docs/plans/2026-10-13-0749-ci-bench-off-pr-plan.md U29 (the plan
says both names are exported; only tailOf has an importer, so the budget
stays private).
* fix(bench): pin Turbo's affected-packages output in the repo scripts lane
Fixes b2fe30db #9 (P1, boundary-testing pin-dependency-semantics, E7):
the plan job's Turbo --packages output was pinned only by a measurement.
scripts/turbo-affected-packages.test.ts runs the plan job's own Turbo
arguments (turboAffectedArgs, now shared with plan-legs.ts) through
node_modules/.bin/turbo, the binary `pnpm exec turbo` resolves to, in
scratch pnpm workspaces, and decodes the output through TurboAffectedPaths,
the schema plan-legs.ts decodes with. It pins three facts: a changed package
reports its repo-relative directory (packages/a), a root file reports "",
and no change reports [].
TurboAffectedPaths moves to turbo-affected.schema.ts: the scripts lane
type-checks with Deno, which cannot resolve bench-plan.schema.ts's in-source
law block (@systemfsoftware/vitest, import.meta.vitest). Its generated codec
laws still run there.
scripts/deno.json takes #319's permission hunk byte for byte, plus an
effect/ import-map entry, and scripts/assert.ts is #319's file byte for
byte, so merging #319 later is a no-op for both. Deno applies the grant
only with -P, which ci.yml:137 does not pass yet; root's workflow owner
adds it in a separate PR. Until then these three tests fail in the Repo
scripts tests step with NotCapable, as #319's do.
Plan: docs/plans/2026-10-13-0954-ci-bench-off-pr-plan.md U31, KD22.
* ci(bench): give each Bench leg job a 20-minute timeout
Root's E10 ruling: per-SHA concurrency groups stay, because every main
SHA must be judged, and each leg job is capped at 20 minutes instead of
60. A leg's longest measured job is 7.68 job-minutes. The plan job keeps 60.
Plan: docs/plans/2026-10-13-0954-ci-bench-off-pr-plan.md U32, KD2.
* docs(plan): record root's resolution of E3-E11 and the 20-minute leg cap
Supersedes the 0749 plan. KD22 records root's rulings of 2026-10-13: #9
fixed by the scripts-lane Turbo pin (U31), inert until ci.yml passes -P in
a separate root-owned PR; E10's 20-minute leg cap (U32, KD2); E8 and E11
closed by A/A run 38375500452; #4 and E4 accepted; E9 satisfied by #335.
* docs(bench): state the 20-minute leg-job timeout in the package description and plan
Fixes the f13b992..aabe4c2 correctness lens finding #2 (P3, confirmed):
cf39161 lowered each Bench leg job's timeout to 20 minutes, but the
@systemfsoftware/stryker-bench description still said GitHub's 60-minute
job timeout is the only kill, and the 0954 plan's CONST-W3 caps bullet, R13
and the read-bench-run admission row still said 60 minutes. The plan job
keeps 60, so plan.ts's step-summary line is unchanged.
Plan: docs/plans/2026-10-13-0954-ci-bench-off-pr-plan.md U32, KD2.
ryanleecode
added a commit
that referenced
this pull request
Oct 13, 2026
… against its own thresholds.break (#299) * test(stryker-js): a merged report below thresholds.break passes the gate R-GATE step 1. Under thresholds.break 100 with every mutant surviving, the unsharded run exits 1, but both shards exit 0 (the shard driver downgrades the break to info), the merge succeeds, and stryker gate exits 0 without score-below-break. Red on main 56426ca by design; the fix lands on this branch. * docs(plan): stryker gate judges a merged report against each project's break * feat(stryker-js): decide a per-project break verdict in a gateScoreBreak workflow R-GATE U1. Each project's score goes through classifyExit, so score == break passes and null break or Unscored gives no verdict (ruling B). A breach fails as ScoreBelowBreak with the score-below-break code and VerdictFail. Plan revised before any unit landed: the merged report's thresholds are reportFromStream's constant (break null), not the first shard's config; per-project thresholds come from each stream's verdict line. * feat(stryker-js)!: stryker gate judges each project of a merged report against its own thresholds.break R-GATE U2-U4. merge records each project's thresholds (from its shard stream's verdict line) and files under a new projects field. gate decodes them through GateReportDocument (decode defaults: break null, projects []), groups files straight-line, and runs gateScoreBreak after the budget check; files outside every project are judged as '.' under the top-level break, which is how an unsharded report is judged. ScoreBelowBreak joins CliFailure (exit 1, score-below-break) and owns its remediation. The shard-run downgrade line names stryker gate. A2 journey: two projects, break 0 and 50, the gate names only the second; it fails (gate exits 0) when the first project's break is applied to both. Plan superseded by the 2115 plan; changeset major. * fix(stryker-js): keep the break verdict under baseline updates, note unrecorded thresholds, ratchet the dogfood break Review rulings on e581c9f (it_bdfe56a0). Update flags write the baselines first and the break check still runs. A merged project with no verdict line in any shard stream records thresholds null; the gate notes it ('no thresholds recorded for this project; re-run its shards with the project's config') instead of passing silently, and both info lines are pinned in the workflow property. New gate journeys: an unsharded report below break under --update-baseline writes the baseline and exits 1 naming '.', and a new survivor fails on the baseline verdict with no break verdict emitted. The shared mutation config's break becomes the ratchet floor 46: main run 38083502908 (56426ca) measured stryker-js 46.45, typescript-checker 54.27, vitest-runner 69.35, e2e-core 72.20 (total 58.27; run 37960922409 totalled 55.01), and every project is judged against the shared break, so 55 would fail stryker-js. Plan superseded by the 2127 plan. * chore(stryker-config): set each dogfood project's thresholds.break to its own measured floor Ruling on 7ec7457 (it_bdfe56a0): a single shared break of 46 let the stronger projects fall far below their scores without failing. Each project's stryker.config.ts now overrides thresholds.break with its score on main run 38083502908 (56426ca), rounded down: stryker-js 46, typescript-checker 54, vitest-runner 69, e2e-core 72. The shared config keeps 46 for projects without an override. The old break of 100 was never enforced in sharded mode. Changeset migration and the README gate row name the project's own config. Plan superseded by the 2220 plan. * chore(changeset): record the dogfood config touch on typescript-checker and vitest-runner as none Changeset Check on fbaae6d named both packages: their stryker.config.ts per-project break floors move the build hash but ship nothing. * fix(stryker-js): keep the dogfood break at 100 and leave every floor to its owner Review P0 on #299 (CONST-E9, CONST-T3, CONST-G3): the change lowered the shared thresholds.break from 100 to 46 and added per-project floors without a CONST-W3 declaration. base.js and the four dogfood stryker.config.ts files now equal main, the floors changeset is deleted, and plan v3 supersedes the 2220 plan: R8 edits no threshold, and any lower floor is the instrument owner's separate, declared change. * docs(plans): state that the dogfood lane enforces the break only after a release pins this gate Review of af63359: plan R8 claimed main's Mutation lane fails at merge, but the lane runs the released CLI pinned by the stryker-published input (mutation.yml:37, package.json:71). The break check reaches the lane only after a release ships this change and the pin moves. Also corrects the mutation.yml line citations, names the now-stale CI comment for Stream K, and marks U5/U6 landed. * test(gate): cover budget-before-break ordering (R5) Finding #2 of the full-path review of PR #299 (run 20261013-032452-bbbb97a3): a gate run that is over the committed budget baseline and also below its break must fail on the budget and emit no score-below-break verdict, so the first failure decides. * test(gate): pin a non-zero score against its break Finding #3 of the full-path review of PR #299 (run 20261013-032452-bbbb97a3): the shared fixture only carried Survived/NoCoverage/Pending mutants, so every printed score was 0.00. Add a report with one Killed and one Survived mutant that scores exactly 50.00: under break 60 it exits 1 naming `scored 50.00 < break 60`, and the same report under break 50 exits 0 because an equal score passes (R2). * docs(changeset): state the info-line condition Finding #4 of the full-path review of PR #299 (run 20261013-032452-bbbb97a3): the changeset promised an info line for a project with no valid mutant unconditionally, but the gate emits it only when that project has a break set. Reword the sentence in consumer voice so it states the condition. * fix(stryker-js): model unrecorded project thresholds as a variant, not null Full-path review 20261013-043442-ce557f97 finding #2, schema-laws/tagged-unions-over-state-by-presence rules 3-4 * fix(stryker-js): log every gate breach line and the next action on failure The exit summary bounds the failure text at 1024 characters (bin/main.ts SPAN_ERROR_LIMIT), so a report with many breaching projects truncated the trailing SCORE_BREAK_REMEDIATION line. Expose the breach lines, notes and remediation as ScoreBelowBreak.lines and log them through Effect.logInfo on the failure path of runScoreBreakGate, mirroring the pass path's cleared.lines logging, so every line reaches stderr uncut while the exit summary is unchanged. Add a built-CLI scenario with 32 projects below break: the composed message exceeds 1024 characters, and stderr still carries every project's score-below-break line and the remediation sentence. Review run 20261013-051057-2f111cb4, finding #6. * test(stryker-js): pin the failing gate message's notes and next action Extend the failure branch of the gate-score-break property law: the composed ScoreBelowBreak.message and the new ScoreBelowBreak.lines each contain the breach line and every expected note, and both end with the remediation sentence (repeated verbatim because SCORE_BREAK_REMEDIATION is module-private). Review run 20261013-051057-2f111cb4, finding #7. * test(stryker-js): drive the merged-report no-verdict gate path through the CLI Add a shard-merge journey: two shards run, both of their mutation-stream verdict lines are stripped before merge (a project is Unrecorded only when no shard of it recorded a verdict), the streams merge, and stryker gate exits 0 naming '. : no thresholds recorded for this project; re-run its shards with the project config'. This drives thresholdsOfProject's Unrecorded branch and the R7 info line end to end, which were covered only by the decision-layer property test. Review run 20261013-051057-2f111cb4, finding #1. * docs(plans): correct the unrecorded-thresholds shape to the shipped tagged union The 0325 plan described unrecorded thresholds as null (U1 thresholds: { break } | null; U2 thresholds: VerdictThresholds | null, 'writes null'; R7 and A3 'thresholds are null'). The shipped contract is a tagged union: ProjectScore.thresholds is ProjectThresholds (Recorded { break } | Unrecorded) and MergedProject.thresholds is MergedProjectThresholds (Recorded VerdictThresholds | Unrecorded), with shard-merge writing Unrecorded. Implementation had started, so the plan moves to a new dated file that supersedes 2026-10-13-0325 and carries the corrected shapes. Review run 20261013-051057-2f111cb4, findings #4 and #5. * fix(stryker-js): compare each project's score to its break inside the gate decision Main's #317 moved score judging out of ClassifyExitCommand (score and breakingThreshold removed) into judge-mutation-score.workflow.ts. The gate decision may not call an imported workflow (dmmf-workflow make-body-purity), so it now keeps the one comparison itself: a breach is a scored project whose percentage is below its recorded break; an equal score passes, as before. * docs(plans): renumber the gate merged-break plan and correct R1-R3/U1 to the shipped gate R1's parenthetical, R2 and U1 named `classifyExit` as the source of the gate's per-project verdict. After #317 the unsharded score flows through `mutationVerdictCell`/`judgeMutationScore`, and the gate decision keeps the same `percentage < threshold` comparison inline (`isBelowItsBreak`, `gate-score-break.workflow.ts:111`) because a decision body may not call an imported workflow. R3 claimed the output starts with a counts-first summary line; on failure `runScoreBreakGate` logs each breach line, the notes and the next action on stderr, and the bounded exit summary carries the counts line. Implementation had started, so the plan moves to a new dated file that supersedes 2026-10-13-0602. Review run 20261013-061137-3bcdcd0e, finding 5. * test(gate): drop the unread breakThreshold parameter from prepareProject prepareProject's third parameter `breakThreshold` was read only by the default `report = reportJsonOf(breakThreshold)`, so every call site that passed a report also passed a break the fixture never read. The signature is now `prepareProject(baseline, budgetBaseline = null, report = reportJsonOf('null'))`, and each caller that wants a break passes the report explicitly (`reportJsonOf(BELOW_BREAK)`, `scoredReportJsonOf('60')` and so on). Behavior and assertions are unchanged: `reportJsonOf(BELOW_BREAK)` builds the same document the default used to. Review run 20261013-061137-3bcdcd0e, finding 9. * fix(stryker-js): require thresholds in the gate report document The gate decoded an absent `thresholds` as `{ break: null }`, so a report that never carried the field read as the verdict "no break". The published report schema (`MutationTestResult.thresholds`) declares it required and Stryker always writes it, so declare it required: a report without it is refused through the gate's undecodable-report path (exit 2), not silently passed. `projects` keeps its decoding default of `[]`, an unsharded report having no projects. Rule schema-laws/tagged-unions-over-state-by-presence; full-path ce-code-review run 20261013-061137-3bcdcd0e finding #1. * fix(stryker-js): build a BreakBreach only for a real breach `BreakBreach` was constructed for every scored project and the consumer filtered with `isBelowItsBreak`, so the type admitted a non-breach. State `percentage < threshold` on the type as a struct check, build a breach only when `breachOf` sees one, and delete the consumer comparison. Behavior is byte-identical for every gate run. The refusal property draws the unrefined base and compares acceptance against the relation written independently. Rule schema-laws/cross-field-invariants-as-struct-checks; full-path ce-code-review run 20261013-061137-3bcdcd0e finding #6. * refactor(stryker-js): count projects with Natural in ScoreBelowBreak `projects` is `command.projects.length`, a non-negative count, which the package idiom types as `S.Natural` (`cap-survivors.workflow.ts:19`), not `S.Int`. Rule schema-laws/invariants-as-refinements; full-path ce-code-review run 20261013-061137-3bcdcd0e finding #8. * docs(plans): supersede the gate merged-break plan for revision 3 Rename the started plan to 2026-10-13-0803 (supersedes the 0732 file, which it replaces) and record revision 3, clearing the three schema-laws findings of full-path ce-code-review run 20261013-061137-3bcdcd0e: A requires `thresholds` in GateReportDocument (tagged-unions-over-state-by-presence, #1), B states `percentage < threshold` on BreakBreach and builds one only for a real breach (cross-field-invariants-as-struct-checks, #6), C counts projects with `S.Natural` (invariants-as-refinements, #8). The changeset notes that a report with no `thresholds` is now refused. * docs(changeset): fit the gate entry under the changelog ceiling d7acf7a added the thresholds-refusal sentence and pushed the body to 1010 bytes, over the 900-byte changelog ceiling the changeset validator enforces. Same consumer-visible content, shorter wording. * fix(stryker-js): gate --update-* help names the break check that still runs Finding #1 (P2, public contract) of ce-code-review run 20261013-113133-02ae7355. The `--update-baseline` and `--update-budget-baseline` help promised "exit 0, instead of gating against it", but `runScoreBreakGate` runs after the baselines are written, so a below-break update-flag run exits 1. Both descriptions now say the baseline is written first and the break check still runs, and `--baseline` names the break verdict beside the survivor verdict. The README rows already stated this; no test or snapshot holds the strings. * docs(plans): supersede the gate merged-break plan for revision 4 Finding #2 (P3) of ce-code-review run 20261013-113133-02ae7355: the plan added by this PR claimed four CLI journeys "at the 2-4 cap" with everything else pushed down to A3/A4, while the branch ships ten CLI scenarios (seven in gate.integration.test.ts, two in shard-merge.integration.test.ts, one in shard-run-verdict.integration.test.ts). Rewrite the Test layers paragraph to enumerate them and say why each is only observable through the command, not a property (A3) or a schema law (A4). Rename the started plan to 2026-10-13-1452 (superseding the 0803 file it replaces) per the plan-discipline rule: implementation had started, so this is a new dated contract, not an in-place edit. It records revision 4 and the finding #1 help rewording as the companion change of the same run. * fix(stryker-js): log each gate breach line once A below-break `stryker gate` failure printed every breach line twice on stderr: `runScoreBreakGate` logs `refused.lines`, and `ScoreBelowBreak.message` embedded the same lines, which the human-mode conclusion printed as the bounded exit summary `exit 1 (VerdictFail): …`. Narrow the message to the counts line only; `lines` keeps the complete uncut detail (breach lines, notes, remediation) the gate logs. Pin the shape in gate.integration.test.ts: each breach line and the remediation line appear exactly once in stderr, and the exit summary still names the count. Update the property assertions so `message` is the counts line while the `lines` assertions stand. Finding #1 of run 20261013-160959-b35ac35e. * docs(plans): supersede the gate merged-break plan for revision 5 Finding #1 of full-path review run 20261013-160959-b35ac35e changed ScoreBelowBreak's message to the counts line only (94aae44). Plan 1452's U1 still said the error prints the same lines inside its message. The started plan is superseded by 1712, which states the counts-only summary and the single logged copy of each breach line. * test(stryker-js): assert the survivor remediation line stays off below-break refusals Finding #2 of review run 20261013-170234-610f3e61 (PR #299). `ownsItsRemediation` keeps the survivor-remediation line ('accept the new survivors with `stryker gate --update-baseline`, or kill them before the next run') off every failure it owns, but no test asserted that absence. Add a locally-written SURVIVOR_REMEDIATION_LINE literal and assert it appears 0 times in stderr for each built-CLI below-break journey that exits 1 via ScoreBelowBreak: the unsharded below-break gate, the `--update-baseline` below-break gate, and the merged many-project below-break gate. * refactor(stryker-js): share one shard-merge-and-gate driver in the integration test Finding #3 of review run 20261013-170234-610f3e61 (PR #299). mergeAndGate and mergeVerdictlessAndGate repeated the same spawn shard 1/2, spawn shard 2/2, merge plan.json into reports/mutation, merge-failure guard, and gate sequence, differing only in the two --out directories and a pre-merge stripVerdictLines step. Extract runShardsMergeAndGate(fixture, { outs, beforeMerge? }) and call it from both; scenario names, assertions, and outcomes are unchanged. mergeWithDowngradedStream stays as-is: it returns the merge outcome instead of gating, so its sequence is not identical. --------- Co-authored-by: Ryan Lee <drdgvhbh@gmail.com>
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
Finding #8 of review run 20261013-170549-4dfc842a (PR #287): the leaf run destructured `since` out of the options and then passed `WholeProject`, so the same unscoped fact was stated twice. `WholeProject` alone now says the leaf run is unscoped; read-project gives a planned diff precedence over `since` and strips `since` on a full scope.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
Finding #8 (run 20261013-203109-28ea241e): tests/__fixtures__/uncovered-block-report.schema.ts declared its own report schema, which the pack schema-laws tests-own-no-schemas law forbids. It is deleted; the integration test decodes with the published Report.MutationTestResult instead (that call-site rewrite is in the shared test file, landed with finding #6).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the inherited
startertemplateREADME.mdwith a complete project README for stryker-js-effect.What Changed
NDJSON) stream protocol emitted bystryker run(stream,phase,plan,tick,verdict,error) and auto-selection of machine mode for non-TTY / agent environments.0Success,1Verdict Failed,2Config Error,3Runtime Error,4Internal Error,128+nProcess Signal).pnpminstall and execution steps (pnpm add -D @systemfsoftware/stryker-js-cli,pnpm exec stryker run) with a samplestryker.config.json.Validation
write-readme-mdcriteria (score: 13/13).pnpm format:checkwithdprint.pnpm gate:tasks(lint, typecheck, tests all clean).