Skip to content

test(stryker-js): draw only valid mutants in the planMutantTests generators - #293

Merged
ryanleecode merged 7 commits into
mainfrom
stream-d/fix-plan-mutant-tests-subsumption
Oct 12, 2026
Merged

ryanleecode merged 7 commits into
mainfrom
stream-d/fix-plan-mutant-tests-subsumption

Conversation

@systemfsoftware-maker

@systemfsoftware-maker systemfsoftware-maker commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Three planMutantTests property laws no longer fail at random. They failed because their generators built Mutant values that the Mutant schema refuses.

Cause

#273 added a struct filter to Mutant (Mutant.schema.ts:144-147): a Subsumed mutant must be Ignored, and a Readmitted mutant must not be. orderedCommandArb and staticKillerCommandArb cleared status but kept the subsumption they drew. Whenever that draw was Subsumed, Mutant.Mutant.make threw before the law ran any step (Schema validation failed, "Failing step: no step ran").

Laws that failed this way:

  • ∀o_OrderedCoveringTests_≡ThePreviousKillerLeadsThenAscendingDryRunTime (orderedCommandArb)
  • ∀r_CoveredRun_≡NeverPublishesKillerTestIds (orderedCommandArb)
  • ∀s_StaticPriorKiller_≡TheRunPublishesItsKnownKillerTestIdsInTheirOrder (staticKillerCommandArb)

Fix

lawfulSubsumptionOf(mutant, status) keeps a drawn subsumption only if it is valid for the status the generator sets. The check is Mutant.subsumptionMatchesStatus, the same predicate the struct filter uses. With no status, only Subsumed draws are dropped. With Ignored, only Readmitted draws are dropped. All three generators use the helper; coverageCommandArb already did this inline. The struct filter, every law's assertions and the run counts are unchanged.

Runs of plan-mutant-tests.workflow.property.test.ts

Tree Runs Passed Failed
main at 3cb134232, before the fix 60 56 4
this branch at 9a4db283b 60 60 0
this branch merged with main at 815754693 30 30 0

On the merged head, pnpm typecheck, oxlint on the changed file, dprint check and gritlint check pass. The changeset is none because no shipped code changes.

Follow-up

  • statusReason gap (pre-existing, on main since cfde427d1). coverageCommandArb always sets the input statusReason to undefined, so ∀r_IgnoredEarlyResult_≡ItsReasonIsForwardedOrNamesARule can't catch a planner that drops a closed mutant's real reason. The generator should draw a valid reason when the mutant is closed. That is out of scope here.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…rators

orderedCommandArb and staticKillerCommandArb clear status but kept the drawn subsumption, so a drawn Subsumed mutant failed Mutant's struct filter (a Subsumed mutant is Ignored, from #273) and the laws fell over before any step ran. Both now keep a drawn subsumption only when it matches the status, through the helper coverageCommandArb already used inline.

@ryanleecode ryanleecode left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at 915911f: up to date with main ce66cd7; check, all 5 e2e legs, checker-parity and Nix green; 0 open threads. Test-only (one property-test generator + changeset 'none'): generators clear subsumption when they clear status, matching Mutant.subsumptionMatchesStatus; the same invalid-draw cause #303 independently measured (5-15 of 2000 draws). Only red: bench typescript-checker leg, failing on every PR until bench leaves the PR path.

@ryanleecode
ryanleecode merged commit cc7fcef into main Oct 12, 2026
26 of 27 checks passed
@ryanleecode
ryanleecode deleted the stream-d/fix-plan-mutant-tests-subsumption branch October 12, 2026 18:05
systemfsoftware-maker added a commit that referenced this pull request Oct 12, 2026
…d fixture bakes, LPT planner

Takes #303's per-project lockfile closure digest, #311's npm-ci fixture
bakes from committed locks, #301's shard planner, and #293's lawful
planMutantTests generators.

- The incremental cache version moves to 7. Main and this branch each moved
  it from 5 to 6 for different layouts, so neither may read the other's
  file. runInputsDigestOf keeps #303's lockfile closure input, and the store
  key reaches it through the same function. #303's closure files, tests and
  oracle fixtures are byte-identical to main.
- incremental-reuse.integration.test.ts keeps this branch's version: #303's
  only edit there was the version number in the incrementalSources scenario
  this branch deletes.
- plan-request.schema.ts keeps #301's PlanShardsRefused and drops the two
  schemas this branch retired.
- The planMutantTests law file takes #293's lawfulSubsumptionOf and drops
  this branch's untestedOf, which fixed the same draw.
- The e2e closure install is per fixture, so only fixtures that request the
  S3 driver install it, and it carries #311's devDependency map. The harness
  stages each fixture's planned dependencies into its manifest, and the bake
  runs npm ci. fixtures:lock regenerated the locks: the verdict-store
  fixture gains its own lock, and the other seven gain the yaml dependency
  #303 added to the engine.
- The docs say a store is shared only between checkouts at the same
  absolute path, because mutant ids hash the absolute file path.
systemfsoftware-maker added a commit that referenced this pull request Oct 12, 2026
…I wiring

Supersedes the verdict-store plan with KTD17. turbo query (2.10.12) shows
the mutation task is cache: false and invalidated per package: a one-file
test change (#293) marks @systemfsoftware/stryker-js#mutation changed. A
turbo task cache cannot carry the store across differing hashes, so the
store keeps actions/cache under the house rule's exception. This layer
adds no task cache, hash cache, change classifier or affected gate. The
.turbo/cache transport on main goes to Stream K's audit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants