Repository navigation
refactor(stryker-js): judge the mutation score in its own workflow and cell - #317
Merged
Merged
Conversation
…re workflows (U3) Sandbox.blueprint.ts (511 lines) now holds only the kind, its withPreprocessor step and its scoped/layer targets (67 lines, one Blueprint.make, no Effect.fn). Its targets run acquireSandboxCell. planSandboxAcquisition decides from inPlace, backupDirectory, symlinkNodeModules and buildCommand: in place with or without a restore, copied with or without node_modules links, and the build command when it is non-empty. acquire-sandbox.cell.ts writes one straight line per variant. The restore stays an Effect.addFinalizer on the caller's scope. Build and link run after it through Cell.gate, never through cell.run. judgeSandboxBuild (sandbox-build.cell.ts) passes a zero exit and fails with the same message main built. classifySandboxDirectory (link-node-modules.cell.ts) skips the temp directory before it matches node_modules, as main's walk did. planNodeModulesLinks links every node_modules the walk found, in order. MakeSandboxInput, SandboxSpec and FilePreprocessor move to Sandbox.schema.ts. Laws for the four workflows draw their production command schemas.
…box acquisition The Sandbox blueprint now runs the acquisition: a stryker.sandbox.acquire span around the prepare cell, the build cell under its own scope, and the link cell. prepare-sandbox.cell.ts (was acquire-sandbox.cell.ts) decides in place or copied under the new stryker.sandbox.prepare span. The link cell runs on every path and logs "Start symlink node_modules" each time, as main does; planNodeModulesLinks decides NodeModulesNotLinked when the sandbox does not link. The build cell's read phase no longer closes a scope; the blueprint scopes each build run. The unused withPreprocessor step and the preprocessors field are gone. The sandbox-acquisition-telemetry suite pins the span tree, step order, symlink log and in-place restore for three paths; it passes on main's code and fails when the build and link steps are swapped or dropped.
Re-grounds U6 on main ddeb67a: refreshed line references (P20), the log texts and exit precedence no suite pins (P21), the four classify-exit survivors re-read from mutation-report-444 (P22), overlap with #298, #271, #297 and #277 with a merge-tree probe (P23), and the cell's span choice (P24, KTD12). U6 no longer stacks on U2, which merged as #268. Raises Q1-Q3 for the conductor before ce-work.
…#297 The plan has executed units (#260, #268, #272), so the change lands as a new dated plan declaring `supersedes:` (harness rule plan-discipline); REPO-D2 caps a PR at one added plan file, which a replacement satisfies. - KTD12: the verdict cell is a Sandwich under a new span, stryker.mutationReporting.judgeScore. ADR-0001's cell row refuses Cell.fromEffect stage wrappers; judging inline in reportAll keeps the verdict's I/O out of a cell (R10). The taxonomy entry edits two files #297 edits, so U6 stacks on U3 (OP13b, P25). - R14: main's spans keep name, attributes and ancestors; a new span is a SpanTaxonomy entry with a cli-contract patch changeset, as #297 did. - U6 adds tests/drain-failure-outranks-verdict.integration.test.ts with a revert proof; P14 rewritten, P23 refreshed, P24 and P25 added.
…d cell U6 of the grab-bag decomposition plan. The score-vs-threshold decision leaves mutation-reporting.service.ts and classify-exit.workflow.ts: - judgeMutationScore (judge-mutation-score.workflow.ts) decides one of five variants, one per verdict log branch on main; each carries a single-valued exitClass, so no arm can return null or undefined. - mutationVerdictCell is a Sandwich under the new span stryker.mutationReporting.judgeScore (KTD12); its write handlers emit main's log lines verbatim and answer the variant's exit class. - classifyExit keeps only precedence over pending classes; every decision variant carries its exitClass, so reportAll's reverse maps go. - reportAll drains, runs the cell, then ranks [verdict, terminalDrain] once; a drain RuntimeError outranks a VerdictFail by precedence. - SpanTaxonomy gains mutationReportingJudgeScore; span-taxonomy.json is regenerated with generate:contract. tests/drain-failure-outranks-verdict.integration.test.ts pins the drain precedence no suite graded (P21): it passes on the pre-change code and fails with the drain class dropped from reportAll's pending list.
Review ruling r29 on U6 (grab-bag decomposition plan, entry 6): - G2: tests/drain-failure-outranks-verdict.integration.test.ts gains a scenario that runs the same in-process mutation run under an InMemorySpanExporter and asserts exactly one stryker.mutationReporting.judgeScore span, parented by stryker.mutationReporting.reportAll, with verdict VerdictFail. With the cell replaced by the bare workflow the scenario fails (parents []). - G4: the judgeMutationScore laws decode the oracle through the ScoreJudgement schema and compare with its schema equivalence instead of JSON.stringify. - F1: the plan says the classifyExit pair and single laws were consolidated into the one pending-array law, and records the span scenario and its revert proof.
Review ruling r30: the plan's U6 Verification grep matched the plan's own mentions of the removed symbols, so it could never return nothing. It now excludes lock files and docs/plans, as earlier plans do, and returns no matches at this head.
…' into stream-d/u6-verdict
Intent r38 (operator rules 2026-10-12): an integration test drives the built CLI or the published API and asserts only what a user observes. - tests/drain-failure-outranks-verdict.integration.test.ts now runs dist/main.mjs on a consumer project with an on-disk reporter plugin (tests/__fixtures__/refusing-reporter/) loaded through `plugins`. One outline: with the refusing plugin the run exits 3, without it 1; each row decodes the stdout verdict record and checks the score is under the break threshold. - The in-process span-parentage scenario is removed: parentage is not in the published span contract. - Plan U6 test lines drop the hand-applied mutant proofs and name the mutant classes as targets for the next main Mutation report.
Debate r39: the span-parentage line named bin/main.ts, but the file is packages/stryker-js/src/bin/main.ts.
…e the verdict decides Review of #317 (run 20261013-032741-afbae46e, validator-confirmed): JudgeMutationScoreCommand carried breakingThreshold as S.NullOr(Percentage) and decide probed its presence, which schema-laws tagged-unions-over-state-by-presence and CONST-D4 forbid. The field now decodes the options' null|Percentage into NoBreakThreshold | BreakAt { threshold } (encode forbidden: the threshold is only ever read from options), and decide dispatches with Match.valueTags. The cell's read is unchanged; laws draw the field's type side.
…s value The command field decoded null|percentage into NoBreakThreshold|BreakAt but refused to encode, so the generated in-source codec laws (∀x_JudgeMutationScoreCommandEnc_=x, ∀x_JudgeMutationScoreCommand_=x) failed on CI run 38344123637. Encode now maps NoBreakThreshold to null and BreakAt to its threshold, the inverse every other decodeTo in this package defines (Plugins.schema.ts).
Review of 9cf18e7 confirmed two P3s: the 1732 plan's U6 approach and test-scenario bullets still described breakingThreshold as S.NullOr(Report.Percentage) with a constructor-defaulted null exitClass. The 0456 plan supersedes section U6 only (as the 2348 plan does for U9), stating BreakThresholdFromOption, the tag dispatch, the decoded-side law draw and the plain S.Null exitClass.
ryanleecode
approved these changes
Oct 13, 2026
ryanleecode
left a comment
ryanleecode
left a comment
Contributor
There was a problem hiding this comment.
Root verdict: d21bd37 green apart from inherited bench; 0 threads; full-path receipt (run 20261013-043138-44a3e4a0) covers both fixes with every matched pack rule ruled and served models named; tagged break threshold fixes the schema-laws finding; integration test drives the built CLI; main delta since base (#290, #321) overlaps only a docs/solutions file and no added code uses #321's renamed members.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
… gate decision Main's #317 moved score judging out of ClassifyExitCommand (score and breakingThreshold removed) into judge-mutation-score.workflow.ts. The gate decision may not call an imported workflow (dmmf-workflow make-body-purity), so it now keeps the one comparison itself: a breach is a scored project whose percentage is below its recorded break; an equal score passes, as before.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
… to the shipped gate R1's parenthetical, R2 and U1 named `classifyExit` as the source of the gate's per-project verdict. After #317 the unsharded score flows through `mutationVerdictCell`/`judgeMutationScore`, and the gate decision keeps the same `percentage < threshold` comparison inline (`isBelowItsBreak`, `gate-score-break.workflow.ts:111`) because a decision body may not call an imported workflow. R3 claimed the output starts with a counts-first summary line; on failure `runScoreBreakGate` logs each breach line, the notes and the next action on stderr, and the bounded exit summary carries the counts line. Implementation had started, so the plan moves to a new dated file that supersedes 2026-10-13-0602. Review run 20261013-061137-3bcdcd0e, finding 5.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
…t of the next run's inputs Takes main through b4db6d8 (#313 root-bound dry-run coverage, #314, #317, #321, #322, #326, #330, #331, #332, #334, #335, #337). Merge resolutions: - dry-run-choice: main's current-root candidate filter over the one incremental file this branch reads; prior statuses come from the verdict store, flaky ids from the kept coverage. - dry-run.cell, deferrable-dry-run.cell, plan-request.cell: pass that one report text into main's choice; the plan prices from the kept coverage. - typescript-checker: main moved the runtime into typescript-checker-runtime.ts; it carries this branch's config/program digest scope. - e2e StrykerCliRunner: main split service from layer; runScript and runInFork move into microsandbox-stryker-cli-runner.ts. - Tests: main's flaky-root CompileError scenario writes the incremental file instead of the removed incrementalSources; the relocated-project multi-report scenario and dry-run-coverage-across-roots go with incrementalSources; the version-gate test uses layerStage. - The solution doc keeps both the store key and the root binding. Fix found while validating the merge: "A run killed after it scored its first mutant spares the next run its initial test run" failed 2 of 4 runs. The fixture kills the CLI at the first mutant event (run/mutant-run.ts:264-265), which can land inside the checkpoint's atomic write of the incremental file (mutation-reporting.service.ts:754). SIGKILL skips the temp file's cleanup (replace-file-atomically.ts:44), so reports/.main.json.<hex>.tmp stays. The crawl excluded only main.json (stryker-outputs.ts, read-project.cell.ts:116-124), so the temp file became a project file and the next run refused its dry run with closureChanged. replace-file-atomically.ts now owns the temp name and the rule matching it, and strykerOutputFilesOf excludes the incremental file's temp siblings. A new scenario plants that temp file between two runs of the built CLI; on the unfixed build the second run executed 2 tests, and it now executes none. The changeset says what holds for a killed run.
ryanleecode
added a commit
that referenced
this pull request
Oct 13, 2026
… against its own thresholds.break (#299) * test(stryker-js): a merged report below thresholds.break passes the gate R-GATE step 1. Under thresholds.break 100 with every mutant surviving, the unsharded run exits 1, but both shards exit 0 (the shard driver downgrades the break to info), the merge succeeds, and stryker gate exits 0 without score-below-break. Red on main 56426ca by design; the fix lands on this branch. * docs(plan): stryker gate judges a merged report against each project's break * feat(stryker-js): decide a per-project break verdict in a gateScoreBreak workflow R-GATE U1. Each project's score goes through classifyExit, so score == break passes and null break or Unscored gives no verdict (ruling B). A breach fails as ScoreBelowBreak with the score-below-break code and VerdictFail. Plan revised before any unit landed: the merged report's thresholds are reportFromStream's constant (break null), not the first shard's config; per-project thresholds come from each stream's verdict line. * feat(stryker-js)!: stryker gate judges each project of a merged report against its own thresholds.break R-GATE U2-U4. merge records each project's thresholds (from its shard stream's verdict line) and files under a new projects field. gate decodes them through GateReportDocument (decode defaults: break null, projects []), groups files straight-line, and runs gateScoreBreak after the budget check; files outside every project are judged as '.' under the top-level break, which is how an unsharded report is judged. ScoreBelowBreak joins CliFailure (exit 1, score-below-break) and owns its remediation. The shard-run downgrade line names stryker gate. A2 journey: two projects, break 0 and 50, the gate names only the second; it fails (gate exits 0) when the first project's break is applied to both. Plan superseded by the 2115 plan; changeset major. * fix(stryker-js): keep the break verdict under baseline updates, note unrecorded thresholds, ratchet the dogfood break Review rulings on e581c9f (it_bdfe56a0). Update flags write the baselines first and the break check still runs. A merged project with no verdict line in any shard stream records thresholds null; the gate notes it ('no thresholds recorded for this project; re-run its shards with the project's config') instead of passing silently, and both info lines are pinned in the workflow property. New gate journeys: an unsharded report below break under --update-baseline writes the baseline and exits 1 naming '.', and a new survivor fails on the baseline verdict with no break verdict emitted. The shared mutation config's break becomes the ratchet floor 46: main run 38083502908 (56426ca) measured stryker-js 46.45, typescript-checker 54.27, vitest-runner 69.35, e2e-core 72.20 (total 58.27; run 37960922409 totalled 55.01), and every project is judged against the shared break, so 55 would fail stryker-js. Plan superseded by the 2127 plan. * chore(stryker-config): set each dogfood project's thresholds.break to its own measured floor Ruling on 7ec7457 (it_bdfe56a0): a single shared break of 46 let the stronger projects fall far below their scores without failing. Each project's stryker.config.ts now overrides thresholds.break with its score on main run 38083502908 (56426ca), rounded down: stryker-js 46, typescript-checker 54, vitest-runner 69, e2e-core 72. The shared config keeps 46 for projects without an override. The old break of 100 was never enforced in sharded mode. Changeset migration and the README gate row name the project's own config. Plan superseded by the 2220 plan. * chore(changeset): record the dogfood config touch on typescript-checker and vitest-runner as none Changeset Check on fbaae6d named both packages: their stryker.config.ts per-project break floors move the build hash but ship nothing. * fix(stryker-js): keep the dogfood break at 100 and leave every floor to its owner Review P0 on #299 (CONST-E9, CONST-T3, CONST-G3): the change lowered the shared thresholds.break from 100 to 46 and added per-project floors without a CONST-W3 declaration. base.js and the four dogfood stryker.config.ts files now equal main, the floors changeset is deleted, and plan v3 supersedes the 2220 plan: R8 edits no threshold, and any lower floor is the instrument owner's separate, declared change. * docs(plans): state that the dogfood lane enforces the break only after a release pins this gate Review of af63359: plan R8 claimed main's Mutation lane fails at merge, but the lane runs the released CLI pinned by the stryker-published input (mutation.yml:37, package.json:71). The break check reaches the lane only after a release ships this change and the pin moves. Also corrects the mutation.yml line citations, names the now-stale CI comment for Stream K, and marks U5/U6 landed. * test(gate): cover budget-before-break ordering (R5) Finding #2 of the full-path review of PR #299 (run 20261013-032452-bbbb97a3): a gate run that is over the committed budget baseline and also below its break must fail on the budget and emit no score-below-break verdict, so the first failure decides. * test(gate): pin a non-zero score against its break Finding #3 of the full-path review of PR #299 (run 20261013-032452-bbbb97a3): the shared fixture only carried Survived/NoCoverage/Pending mutants, so every printed score was 0.00. Add a report with one Killed and one Survived mutant that scores exactly 50.00: under break 60 it exits 1 naming `scored 50.00 < break 60`, and the same report under break 50 exits 0 because an equal score passes (R2). * docs(changeset): state the info-line condition Finding #4 of the full-path review of PR #299 (run 20261013-032452-bbbb97a3): the changeset promised an info line for a project with no valid mutant unconditionally, but the gate emits it only when that project has a break set. Reword the sentence in consumer voice so it states the condition. * fix(stryker-js): model unrecorded project thresholds as a variant, not null Full-path review 20261013-043442-ce557f97 finding #2, schema-laws/tagged-unions-over-state-by-presence rules 3-4 * fix(stryker-js): log every gate breach line and the next action on failure The exit summary bounds the failure text at 1024 characters (bin/main.ts SPAN_ERROR_LIMIT), so a report with many breaching projects truncated the trailing SCORE_BREAK_REMEDIATION line. Expose the breach lines, notes and remediation as ScoreBelowBreak.lines and log them through Effect.logInfo on the failure path of runScoreBreakGate, mirroring the pass path's cleared.lines logging, so every line reaches stderr uncut while the exit summary is unchanged. Add a built-CLI scenario with 32 projects below break: the composed message exceeds 1024 characters, and stderr still carries every project's score-below-break line and the remediation sentence. Review run 20261013-051057-2f111cb4, finding #6. * test(stryker-js): pin the failing gate message's notes and next action Extend the failure branch of the gate-score-break property law: the composed ScoreBelowBreak.message and the new ScoreBelowBreak.lines each contain the breach line and every expected note, and both end with the remediation sentence (repeated verbatim because SCORE_BREAK_REMEDIATION is module-private). Review run 20261013-051057-2f111cb4, finding #7. * test(stryker-js): drive the merged-report no-verdict gate path through the CLI Add a shard-merge journey: two shards run, both of their mutation-stream verdict lines are stripped before merge (a project is Unrecorded only when no shard of it recorded a verdict), the streams merge, and stryker gate exits 0 naming '. : no thresholds recorded for this project; re-run its shards with the project config'. This drives thresholdsOfProject's Unrecorded branch and the R7 info line end to end, which were covered only by the decision-layer property test. Review run 20261013-051057-2f111cb4, finding #1. * docs(plans): correct the unrecorded-thresholds shape to the shipped tagged union The 0325 plan described unrecorded thresholds as null (U1 thresholds: { break } | null; U2 thresholds: VerdictThresholds | null, 'writes null'; R7 and A3 'thresholds are null'). The shipped contract is a tagged union: ProjectScore.thresholds is ProjectThresholds (Recorded { break } | Unrecorded) and MergedProject.thresholds is MergedProjectThresholds (Recorded VerdictThresholds | Unrecorded), with shard-merge writing Unrecorded. Implementation had started, so the plan moves to a new dated file that supersedes 2026-10-13-0325 and carries the corrected shapes. Review run 20261013-051057-2f111cb4, findings #4 and #5. * fix(stryker-js): compare each project's score to its break inside the gate decision Main's #317 moved score judging out of ClassifyExitCommand (score and breakingThreshold removed) into judge-mutation-score.workflow.ts. The gate decision may not call an imported workflow (dmmf-workflow make-body-purity), so it now keeps the one comparison itself: a breach is a scored project whose percentage is below its recorded break; an equal score passes, as before. * docs(plans): renumber the gate merged-break plan and correct R1-R3/U1 to the shipped gate R1's parenthetical, R2 and U1 named `classifyExit` as the source of the gate's per-project verdict. After #317 the unsharded score flows through `mutationVerdictCell`/`judgeMutationScore`, and the gate decision keeps the same `percentage < threshold` comparison inline (`isBelowItsBreak`, `gate-score-break.workflow.ts:111`) because a decision body may not call an imported workflow. R3 claimed the output starts with a counts-first summary line; on failure `runScoreBreakGate` logs each breach line, the notes and the next action on stderr, and the bounded exit summary carries the counts line. Implementation had started, so the plan moves to a new dated file that supersedes 2026-10-13-0602. Review run 20261013-061137-3bcdcd0e, finding 5. * test(gate): drop the unread breakThreshold parameter from prepareProject prepareProject's third parameter `breakThreshold` was read only by the default `report = reportJsonOf(breakThreshold)`, so every call site that passed a report also passed a break the fixture never read. The signature is now `prepareProject(baseline, budgetBaseline = null, report = reportJsonOf('null'))`, and each caller that wants a break passes the report explicitly (`reportJsonOf(BELOW_BREAK)`, `scoredReportJsonOf('60')` and so on). Behavior and assertions are unchanged: `reportJsonOf(BELOW_BREAK)` builds the same document the default used to. Review run 20261013-061137-3bcdcd0e, finding 9. * fix(stryker-js): require thresholds in the gate report document The gate decoded an absent `thresholds` as `{ break: null }`, so a report that never carried the field read as the verdict "no break". The published report schema (`MutationTestResult.thresholds`) declares it required and Stryker always writes it, so declare it required: a report without it is refused through the gate's undecodable-report path (exit 2), not silently passed. `projects` keeps its decoding default of `[]`, an unsharded report having no projects. Rule schema-laws/tagged-unions-over-state-by-presence; full-path ce-code-review run 20261013-061137-3bcdcd0e finding #1. * fix(stryker-js): build a BreakBreach only for a real breach `BreakBreach` was constructed for every scored project and the consumer filtered with `isBelowItsBreak`, so the type admitted a non-breach. State `percentage < threshold` on the type as a struct check, build a breach only when `breachOf` sees one, and delete the consumer comparison. Behavior is byte-identical for every gate run. The refusal property draws the unrefined base and compares acceptance against the relation written independently. Rule schema-laws/cross-field-invariants-as-struct-checks; full-path ce-code-review run 20261013-061137-3bcdcd0e finding #6. * refactor(stryker-js): count projects with Natural in ScoreBelowBreak `projects` is `command.projects.length`, a non-negative count, which the package idiom types as `S.Natural` (`cap-survivors.workflow.ts:19`), not `S.Int`. Rule schema-laws/invariants-as-refinements; full-path ce-code-review run 20261013-061137-3bcdcd0e finding #8. * docs(plans): supersede the gate merged-break plan for revision 3 Rename the started plan to 2026-10-13-0803 (supersedes the 0732 file, which it replaces) and record revision 3, clearing the three schema-laws findings of full-path ce-code-review run 20261013-061137-3bcdcd0e: A requires `thresholds` in GateReportDocument (tagged-unions-over-state-by-presence, #1), B states `percentage < threshold` on BreakBreach and builds one only for a real breach (cross-field-invariants-as-struct-checks, #6), C counts projects with `S.Natural` (invariants-as-refinements, #8). The changeset notes that a report with no `thresholds` is now refused. * docs(changeset): fit the gate entry under the changelog ceiling d7acf7a added the thresholds-refusal sentence and pushed the body to 1010 bytes, over the 900-byte changelog ceiling the changeset validator enforces. Same consumer-visible content, shorter wording. * fix(stryker-js): gate --update-* help names the break check that still runs Finding #1 (P2, public contract) of ce-code-review run 20261013-113133-02ae7355. The `--update-baseline` and `--update-budget-baseline` help promised "exit 0, instead of gating against it", but `runScoreBreakGate` runs after the baselines are written, so a below-break update-flag run exits 1. Both descriptions now say the baseline is written first and the break check still runs, and `--baseline` names the break verdict beside the survivor verdict. The README rows already stated this; no test or snapshot holds the strings. * docs(plans): supersede the gate merged-break plan for revision 4 Finding #2 (P3) of ce-code-review run 20261013-113133-02ae7355: the plan added by this PR claimed four CLI journeys "at the 2-4 cap" with everything else pushed down to A3/A4, while the branch ships ten CLI scenarios (seven in gate.integration.test.ts, two in shard-merge.integration.test.ts, one in shard-run-verdict.integration.test.ts). Rewrite the Test layers paragraph to enumerate them and say why each is only observable through the command, not a property (A3) or a schema law (A4). Rename the started plan to 2026-10-13-1452 (superseding the 0803 file it replaces) per the plan-discipline rule: implementation had started, so this is a new dated contract, not an in-place edit. It records revision 4 and the finding #1 help rewording as the companion change of the same run. * fix(stryker-js): log each gate breach line once A below-break `stryker gate` failure printed every breach line twice on stderr: `runScoreBreakGate` logs `refused.lines`, and `ScoreBelowBreak.message` embedded the same lines, which the human-mode conclusion printed as the bounded exit summary `exit 1 (VerdictFail): …`. Narrow the message to the counts line only; `lines` keeps the complete uncut detail (breach lines, notes, remediation) the gate logs. Pin the shape in gate.integration.test.ts: each breach line and the remediation line appear exactly once in stderr, and the exit summary still names the count. Update the property assertions so `message` is the counts line while the `lines` assertions stand. Finding #1 of run 20261013-160959-b35ac35e. * docs(plans): supersede the gate merged-break plan for revision 5 Finding #1 of full-path review run 20261013-160959-b35ac35e changed ScoreBelowBreak's message to the counts line only (94aae44). Plan 1452's U1 still said the error prints the same lines inside its message. The started plan is superseded by 1712, which states the counts-only summary and the single logged copy of each breach line. * test(stryker-js): assert the survivor remediation line stays off below-break refusals Finding #2 of review run 20261013-170234-610f3e61 (PR #299). `ownsItsRemediation` keeps the survivor-remediation line ('accept the new survivors with `stryker gate --update-baseline`, or kill them before the next run') off every failure it owns, but no test asserted that absence. Add a locally-written SURVIVOR_REMEDIATION_LINE literal and assert it appears 0 times in stderr for each built-CLI below-break journey that exits 1 via ScoreBelowBreak: the unsharded below-break gate, the `--update-baseline` below-break gate, and the merged many-project below-break gate. * refactor(stryker-js): share one shard-merge-and-gate driver in the integration test Finding #3 of review run 20261013-170234-610f3e61 (PR #299). mergeAndGate and mergeVerdictlessAndGate repeated the same spawn shard 1/2, spawn shard 2/2, merge plan.json into reports/mutation, merge-failure guard, and gate sequence, differing only in the two --out directories and a pre-merge stripVerdictLines step. Extract runShardsMergeAndGate(fixture, { outs, beforeMerge? }) and call it from both; scenario names, assertions, and outcomes are unchanged. mergeWithDowngradedStream stays as-is: it returns the merge outcome instead of gating, so its sequence is not identical. --------- Co-authored-by: Ryan Lee <drdgvhbh@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unit U6 of
docs/plans/2026-10-12-1732-refactor-grab-bag-decomposition-plan.md. It was built as a layer on #297, which has merged; main (with #297 squashed as c8bdd7f) is merged in, so this PR targetsmainand its diff is U6 only.What changes
The decision of whether a run's mutation score breaks
thresholds.breakleavesmutation-reporting.service.tsand becomes:judgeMutationScore(src/judge-mutation-score.workflow.ts): a pure workflow with five judgement variants, each carrying the exit class it implies (VerdictFailonly for a scored run strictly below a set threshold).mutationVerdictCell(src/mutation-verdict.cell.ts): aSandwichunder the new spanstryker.mutationReporting.judgeScore. Its write handlers emit the same log lines as before.classifyExitkeeps only precedence: the most severe class among the pending ones. Its command no longer carries the score.NoBreakThreshold | BreakAt), decoded from the options'null | Percentage, so the decision dispatches on a tag and never on whether a field is present.reportAllcalls the cell, and a reporter-drainRuntimeErrorstill outranksVerdictFail. The span is added toSpanTaxonomyandcontract/span-taxonomy.json(regenerated withgenerate:contract). Patch changesets forstryker-jsandstryker-js-cli-contract.Behaviour kept
break-threshold-exitfor 1 below the threshold and 0 with none;drain-failure-outranks-verdictfor 3 and 1. Equal and unscored are graded by the judge property laws.)Tests
Evidence classes used below: CI report (main's Mutation report), real-input argument (why a test fails on a plausible bug, from its inputs and oracle), local check (run in this worktree). No claim rests on a hand-applied mutant; the earlier revert and cell-swap proofs are withdrawn.
src/__tests__/judge-mutation-score.workflow.property.test.ts(pure-core property test beside the workflow):S.toEquivalence(ScoreJudgement)against an oracle that usesMath.sign, not<. Real-input argument: a null threshold read as 0, an unscored run judged against a threshold, or an arm answeringundefinedeach yields a value the oracle's variant does not equal.Scored { p }against thresholdp) passes. Real-input argument: independent draws almost never tie, so this law is the one that targets<→<=. CI report: not yet graded; the firstmainMutation report after merge (the nightly run, or a dispatch by Stream K or the root) says whether that mutant dies.src/__tests__/classify-exit.workflow.property.test.ts: one law over drawn pending arrays (empty, single, many) replaces the old pair and single laws. Real-input argument: the oracle is the test's own severity table (codeOf/worseOf), so a reordered precedence or a dropped element changes the answer for some drawn array.tests/drain-failure-outranks-verdict.integration.test.ts(rewritten for the 2026-10-12 integration-test rule). See "Integration test compliance".Integration test compliance
drain-failure-outranks-verdict.integration.test.tsnow drives the built CLI (dist/main.mjs run) on a consumer project, asbreak-threshold-exit.integration.test.tsalready does. The refusing reporter is a reporter plugin on disk (tests/__fixtures__/refusing-reporter/index.mjs+worker.mjs), loaded the way a user loads one:plugins: [<file URL>]andreporters: ['json', 'refusing']instryker.config.mjs. Its worker refuses the batch that carries the finished report.One scenario outline, two rows, same project (tests kill no mutant, break threshold 100):
jsonand the refusing pluginverdictrecord on stdout (decoded with the publishedRunEvent.VerdictReached) scores the run under the thresholdjsononlyIt asserts nothing internal: no
MutationTestDonefield, no span, no engine service. Exit-class codes are the published table instryker-js-plugin-interface/src/ExitClass.schema.ts:11-16. Real-input argument: ifreportAllstopped putting the drain class in the pending list, row 1 would exit 1; if the drain failure stopped being counted, the same; if the verdict were never reached, the score check fails. Local check: both rows pass; a scratch run of row 1's project loggedReporter "refusing" failed while draining the terminal reportbefore exiting 3, so the plugin loaded and the refusal is what decided the exit.The in-process span scenario is removed. Span parentage is not a published contract: each
contract/span-taxonomy.jsonentry carries onlyid,nameandattributes(stryker-js-cli-contract/scripts/contract-documents.ts:23-35), and the CLI exports spans only throughOTEL_*variables that no user doc mentions (packages/stryker-js/src/bin/main.ts:121-138). ThejudgeScorespan name is still published in the taxonomy.Where the reporter plugin lives
tests/__fixtures__/refusing-reporter/{index,worker}.mjs, kept there on purpose; the root's #272 ban on__fixtures__(r14) is recorded as conflicting with the gate that enforces placement:__fixtures__.@systemfsoftware/oxlint-plugin-test-disciplineruletests-dir-helpers-in-fixtures(inoxlint-config-recommended, whichpackages/stryker-js/oxlint.config.ts:1extends) reports any non-test module undertests/outsidetests/__fixtures__/as an error. A probe in this worktree, deleted afterwards:tests/refusing-reporter-plugin/probe.ts→probe.ts is forbidden … Fix: move it under tests/__fixtures__/, exit 1; the same file undertests/__fixtures__/→ clean. A job-named directory undertests/therefore failspnpm lint.**/*.mjsin every preset this package uses (oxlint-config-dmmfandoxlint-config-cell-architectureignorePatterns), soworker.mjsis unlinted wherever it sits. Thegritlint.json:15ignore does not matter here either: its three packs (gritlint.json:2-14: source-resolution, typecheck-build-mode, npm-provenance) check tsconfig, vitest config and package manifest edges, none of which a worker module has. The relaxations inpackages/toolchain/oxlint-ignorer-config/lib/base.js:102-115apply only to packages that extend that config (the ignorer and framework packages), not tostryker-js.main. Every on-disk plugin onmainis atests/__fixtures__/<job>/{index,worker}.mjspair (cost-checker,deaf-runner,frozen-runner,plugin-runner, …), andmainhas no non-test file underpackages/stryker-js/tests/outside__fixtures__/(git ls-tree -r origin/main -- packages/stryker-js/tests).file:URL and spawnsworker.mjsas its own process, which resolves its imports from where the file sits.The two rules conflict for any on-disk plugin. Resolving that (an exemption in the lint rule, or a sanctioned non-
__fixtures__home) belongs to the root, not to this PR.Mutation evidence
CI report: main run 38239240750 (
mutation-report-444). This PR deletes the code of surviving mutants2c6daf224e727981,354e6d1d3c4dc374,5f1b04c8d61df244and6cea2aa080723204(reported as deleted). The new laws target two mutant classes injudge-mutation-score.workflow.ts, an arm returningundefinedand<→<=. Those are targets until the firstmainMutation report after merge grades them (the nightly run, or a dispatch by Stream K or the root). No local mutation run of any kind was made.Review receipt (reviewed head
9cf18e7d0; fixd21bd37fb; current headd21bd37fb)ce-code-review mode:agent base:origin/main, full path, report-only, run20261013-043138-44a3e4a0, base251201755, reviewed head9cf18e7d0. It reviews both fixes since round 1:bdd20dea5(the break threshold decodes intoNoBreakThreshold | BreakAt) and9cf18e7d0(that codec encodes back, because CI run 38344123637 failed the generated in-source codec laws∀x_JudgeMutationScoreCommandEnc_=xand∀x_JudgeMutationScoreCommand_=x). Round 1's local run missed those laws because it named only the__tests__file; this round ran the package's full suite (280 files, 1731 tests) from clean property seeds, plus typecheck, lint,lint:tsgoand the build. Verdict on9cf18e7d0: Ready with fixes. Both fixes are ind21bd37fb, a docs-only commit that has had no lens of its own.Lenses and served models (read from each agent's transcript; the configured MiniMax M3.1 Flash Preview served no turn this round):
correctness-reviewergeneralist-reviewergeneralist-reviewergeneralist-reviewergeneralist-revieweradversarial-reviewergeneralist-reviewercorrectness-reviewerNo xai or Grok model appears in any transcript. Merge and report synthesis ran on
docs-engineer, which reviews nothing itself. No lens timed out.Pack rules (enrolled by
.compound-engineering/config.yaml:1-4: cell-architecture, boundary-testing, schema-laws; judged against today's laws: integration tests drive the built CLI, mutation selects*.workflow.tsonly). 13 comply, 0 violate on9cf18e7d0. Round 1's violation (schema-laws/tagged-unions-over-state-by-presence) now complies atjudge-mutation-score.workflow.ts:16-30.9cf18e7d0cell-architecture/sandwich-phase-order.mdpackages/stryker-js/src/mutation-verdict.cell.ts:20-24cell-architecture/pure-decision-workflows.mdpackages/stryker-js/src/judge-mutation-score.workflow.ts:92-106cell-architecture/four-channel-contracts.mdpackages/stryker-js/src/mutation-verdict.cell.ts:20-66cell-architecture/service-and-layer-boundaries.mdpackages/stryker-js/src/mutation-reporting.service.ts:879cell-architecture/decode-never-cast.mdpackages/stryker-js/src/mutation-verdict.cell.ts:13-18boundary-testing/no-mocks-on-internal-glue.mdpackages/stryker-js/tests/drain-failure-outranks-verdict.integration.test.ts:95-127boundary-testing/real-system-oracles.mdpackages/stryker-js/tests/drain-failure-outranks-verdict.integration.test.ts:98-127schema-laws/tagged-unions-over-state-by-presence.mdpackages/stryker-js/src/judge-mutation-score.workflow.ts:16-30schema-laws/data-only-schema-classes.mdpackages/stryker-js/src/judge-mutation-score.workflow.ts:31-83schema-laws/law-failure-is-a-codec-defect.mdpackages/stryker-js/src/judge-mutation-score.workflow.ts:24-28schema-laws/tests-own-no-schemas.mdpackages/stryker-js/src/__tests__/judge-mutation-score.workflow.property.test.ts:50-51schema-laws/invariants-as-refinements.mdpackages/stryker-js/src/judge-mutation-score.workflow.ts:58,65schema-laws/cross-field-invariants-as-struct-checks.mdpackages/stryker-js/src/judge-mutation-score.workflow.ts:55-61Round 2 findings and dispositions:
breakingThreshold: S.NullOr(Report.Percentage)and a constructor-defaulted nullexitClass(2026-10-12-1732plan:471)d21bd37fb:docs/plans/2026-10-13-0456-refactor-mutation-verdict-plan.mdsupersedes section U6 (as the2348plan does for U9) and statesBreakThresholdFromOption, the tag dispatch and the plainS.Null. The 1732 plan is not rewritten in place (plan-discipline: a started plan is superseded).S.NullOr(Report.Percentage)and the threshold's presence (:482-483)d21bd37fb(same plan): the law drawsS.toType(JudgeMutationScoreCommand.fields.breakingThreshold), and the codec's round trip belongs to the generated in-source laws.Correctness, project-standards, api-contract and adversarial: no findings. Residual risks kept as notes:
CommandRejectedanswersInternalError(exit 4) and is unreachable whilethresholds.breakdecodes asNullOr(Percentage).judgeScorespan's emission and parentage are not a published contract.Round 1 findings and dispositions (head
3348d640d, run20261013-032741-afbae46e; 12 comply, 1 violate):NullOrpresence for the threshold statebdd20dea5(the threshold decodes into `NoBreakThresholdempty-file-selection.integration.test.ts(onmain) runsUnscoredAgainstThresholdwithbreak: 100and pins its null exit class. Only the handler's log text is unasserted, see "Behaviour kept".judgeScorespan is asserted by no testmain.main; extracting a shared harness is its own unit, and the lint rule fixes its home undertests/__fixtures__/.CommandRejectedexits 4 without logging the issueNullOr(Percentage)before the cell.Correctness and api-contract: no findings. Their residual risks are kept as notes:
CommandRejectedis unreachable; span parentage is not a published contract; the contract change is additive.Kept from the earlier round, the CONST-D2 rejection. That round's validator confirmed a P2 claiming the exit decision carries
exitClassas a field thatreportAllreads, against CONST-D2's "callers branch on the variant tag" check. It is rejected as a false positive:The P2 is rejected as a false positive. CONST-D2 forbids telling failures apart by a field; here every failure keeps its own tagged variant, and each variant's
exitClassis single-valued by schema, so it adds no distinction the tag does not fix. The branching consumer (the cell's write handlers) dispatches on the tag;reportAllonly forwards the value asMutationTestDone.verdict: Plugin.ExitClass | null. Lint, CONST-D2's gate, passes. The suggested alternative, a tag-to-class map inreportAll, would sit in an unmutated service file, while today each variant's class is graded by theclassifyExitlaw against an independent severity oracle.Delta review:
23a7e0b30..9cf18e7d0The rounds above cover
3348d640d(run20261013-032741-afbae46e),bdd20dea5and23a7e0b30; the round-2 run20261013-043138-44a3e4a0read9cf18e7d0's tree with six other lenses. This section is the root's requested single-lens check of exactly the delta9cf18e7d0adds over23a7e0b30: one file,packages/stryker-js/src/judge-mutation-score.workflow.ts(+6/−1), where the break-threshold codec's encode changes fromSchemaGetter.forbiddentoNoBreakThreshold → null,BreakAt → threshold.23a7e0b30..9cf18e7d0(one commit).20261013-050652-7694f5dc, onecorrectness-reviewerlens, report-only.BreakAtthreshold outside 0-100. Such a value only type-checks when built by hand (the variant structs are module-private); no in-repo path builds one.Match.valueTagsthrows aTypeError, not a schema issue, if a non-variant value reaches the encode. That input is a type error at every call site.effect-cell-types12.0.0 decodes commands and encodes only decisions and errors. So exit codes and verdict log lines cannot change through this delta.0boundary and thenull ↔ NoBreakThresholdwire value are pinned only by the generated in-source laws, not by a named-value assertion.Branch head:
d21bd37fb, a docs-only plan commit (docs/plans/2026-10-13-0456-refactor-mutation-verdict-plan.md) pushed after9cf18e7d0. This delta review does not cover it. CI ond21bd37fbat one read: Nix 38348927279, Bench 38348927357, Changeset Check 38348927557 and Commitlint 38348927287 passed; CI 38348927324 was still in progress.Build and CI scope
No Turborepo task-cache or CI-scope behavior changes. None of the changed files is
turbo.json, apackage.json,pnpm-lock.yaml, a workflow under.github/, a hook, a script, a Nix file, or a vitest/tsdown config; no task inputs, outputs or filters move. The new reporter plugin fixture sits underpackages/stryker-js/tests/__fixtures__/, which the existing test task already covers. Product incremental reuse is untouched:reportAll's incremental-report path is unchanged. The turbo-query-optimization rule therefore does not apply to this unit.