Skip to content

feat(repo): plan deterministic LPT mutation shards with stryker plan - #187

Merged
kiro-systemf[bot] merged 5 commits into
mainfrom
stryker/sota-2-plan
Oct 6, 2026
Merged

kiro-systemf[bot] merged 5 commits into
mainfrom
stryker/sota-2-plan

Conversation

@systemfsoftware-maker

@systemfsoftware-maker systemfsoftware-maker commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

stryker plan --target-seconds N [--max-shards K] [--projects <dir>...] [--out <file>] [--full] discovers each project's mutants without running tests, decides reuse against that project's incremental report, costs every scheduled mutant, and packs mutants across shards with longest-processing-time (LPT) list scheduling. It emits a versioned ShardPlan (project mutant lists, predicted seconds, GitHub Actions matrix) on the new plan stream event, deterministic for the same inputs. ShardPlan is a new minor contract of @systemfsoftware/stryker-js-cli-contract.

This branch also closes the flaky property the plan's full turbo test run exposed:

  • ∀c_Mutants_≡LptKeepsTheBusiestShardWithinFourThirdsOfAveragePlusLargestItem falsified at seed 6: three mutants costing 3.4927206787701224e293 ms, 1.7976931348623123e308 ms and 1.8961673493674487e-132 ms with maxShards: 1 sum to Infinity; PlannedShard.make refuses a non-finite predictedSeconds, so planShards threw Error: Schema validation failed instead of returning a plan.
  • costMs is now bounded to Number.MAX_SAFE_INTEGER ms, so any shard total over a realizable mutant list stays a finite double and every accepted command has a representable plan. The LPT bound itself is untouched; the in-source refusal law pins the boundary with the falsifying cost.

origin/main's worker line already declares stream schema version 4.0, so this branch raises StreamSchemaVersion to 5.0 and regenerates packages/stryker-js-cli-contract/contract/stream.schema.json; the wire fixtures move with it.

Borrowed

  • cargo-mutants: a mutation run shards over mutant ids, not packages
  • LPT list scheduling (Graham 1969): sort by cost descending, place each mutant on the least-loaded shard, which keeps the busiest shard within avg + (1 - 1/m) * largest
  • Bazel: deterministic action output — the same inputs produce byte-identical plan JSON, so shards are reproducible and cache-friendly

Rejected

  • index slicing (mutants[i::n] or chunked id lists): blind to the cost model, it parks the expensive mutants on one shard and makes the wall clock the max instead of the mean

Failing before (recorded)

  1. The reported property, reproduced deterministically with the same generator and predicate (seed 6, runs 100) and shrunk to:
PlanShardsCommand {
  targetSeconds: 5e-324,
  maxShards: 1,
  mutants: [
    { project: "", id: "0000000000000000", costMs: 3.4927206787701224e+293 },
    { project: " ", id: "3bd10c5abb0509cc", costMs: 1.7976931348623123e+308 },
    { project: "", id: "5f6489e791d67f96", costMs: 1.8961673493674487e-132 },
  ],
}

planShards threw Error: Schema validation failed on it (Infinity shard total -> predictedSeconds rejected).

  1. The new in-source refusal law ∀c_CostMsRefusal_≡RepresentableCostMs failed against the unbounded schema:
FAIL  src/plan-shards.workflow.ts > ∀c_CostMsRefusal_≡RepresentableCostMs
PropertyRefuted: ∀c_CostMsRefusal_≡RepresentableCostMs: the property was falsified. Property falsified after 1 run(s) and 1 shrink(s)
  Shrunk input: [0]
  Failure: returned false
  Replay: [0,"1718690611394657",0,0,[0],"ReturnedFalse"]

Gates (this worktree, rebased onto origin/main 5289c07)

command result
stryker-js vitest run src/plan-shards.workflow.ts src/__tests__/plan-shards.workflow.property.test.ts 11 passed (6 generated codec laws + refusal law + 4 properties)
stryker-js vitest run tests/plan.integration.test.ts (after build) 1 passed
stryker-js vitest run tests/run-event-stream.integration.test.ts tests/framework-stream.integration.test.ts 11 passed
cli-contract vitest run tests/run-event-wire-line.integration.test.ts 5 passed
seed scan (3000 seeds x 100 runs, recorded predicate) no falsification; the recorded command refused
derived-arbitrary audit (2000 draws) every costMs <= 2^53-1
pnpm build (cli-contract, stryker-js; includes typecheck) pass
pnpm lint (both packages), pnpm lint:tsgo 0 errors
pnpm format:check, pnpm guard:projects pass
scripts/guards/check-contract-versions.ts compared 4 contract documents, pass
scripts/guards/check-verdict-semantics.ts declaration: unchanged

Mutation runs are never executed locally (engine refusal); the plan unit suites use fake/small fixtures.

@systemfsoftware-maker systemfsoftware-maker changed the title stryker/sota 2 plan feat(repo): plan deterministic LPT mutation shards with stryker plan Oct 6, 2026
Add the ShardPlan contract (version 1) to @systemfsoftware/stryker-js-cli-contract and a new `stryker plan` subcommand that discovers each project's mutants without running tests, decides reuse through the existing incremental-diff workflow, costs every scheduled mutant, and packs mutants across shards with a longest-processing-time heuristic. Emit the plan on the `plan` stream event, which moves the stream schema version to 4.0

Verdict-Semantics: unchanged
Replace the shard plan's S.Number fields with S.Finite so effect-tsgo's schemaNumber diagnostic and the RunEventWireLine round-trip law hold without a suppression. Regenerate the stream contract document and the API report

Verdict-Semantics: unchanged
The property ∀c_Mutants_≡LptKeepsTheBusiestShardWithinFourThirdsOfAveragePlusLargestItem falsified at seed 6: costs near Number.MAX_VALUE summed to Infinity, PlannedShard.make refused the non-finite predictedSeconds, and planShards threw Error: Schema validation failed instead of returning a plan. Bound costMs to 2^53-1 ms so every shard total over a realizable mutant list stays a finite double, and pin the refusal boundary with the falsifying cost in an in-source law

Verdict-Semantics: unchanged
origin/main's worker line already declares 4.0, so the plan event added by this branch has to sit above it. Raise the literal, regenerate the stream contract document, move the wire fixtures to the new version, and restate the new version in the contract changeset

Verdict-Semantics: unchanged
The plan-stream commit raised StreamSchemaVersion from 4.0 to 5.0 and updated
the unit literals, but failing-run.e2e.test.ts still pinned the dry-run error
document to 4.0, so e2e rest-1 failed strict equality against the packed CLI's
5.0

Verdict-Semantics: unchanged

@kiro-systemf kiro-systemf Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed by Kiro: SOTA-ENGINE track 2: stryker plan packs mutants into deterministic LPT shards from the recorded cost model; getOrThrowWith only in an integration test. No threshold, exclude or scope change. All checks green on this head; merges cleanly after #183.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant