Repository navigation
perf(stryker-js): enter the instrument phase before instrumenting (T10 U1) - #348
Merged
Merged
Conversation
The `instrument` phase mark and its `instrument` span ran in the instrument cells' write phase, after `instrumentFiles` and `makeSandbox` had already run in the read phase. Instrumentation and the sandbox copy therefore counted under `prepare`, and every Bench sample read `instrument` as 0-1 ms, so the instrument share of wall clock that gates the T9 native-core restart could not be read. Both cells (`instrument.cell.ts` for runs, `plan-instrument.cell.ts` for plans) now open the phase span and emit the mark at the start of the read phase; the write phase only offers file skips. The new engine integration test drives a dry-run-only run and a plan through the published `Engine` entry points and asserts from OTel span intervals that the gather and sandbox acquisition spans fall inside the single `instrument` span and that the phase mark ends before gathering starts. It failed on the old placement and passes on the new one. Lands the T10 infection-filtered execution plan (U1 of it, R18, KTD9).
Root's reading of the first four green main Bench push runs after #316 (38429923569, 38430066932, 38436282117, 38436508210) replaces the plan's [unverified] baseline run ids, with the per-leg median-of-four phase table. Those runs read `instrument` as about 0 ms on every leg because the mark is misplaced, so they are the baseline later T10 layers compare against, not the T9 reading: R18 now reads `S` from the first four green main push runs after U1 merges. R17 states Bench's non-overlapping-samples rule, since a single typescript-checker run took 33.5 s against about 46 s for the other three.
Review finding (P3, testing): the changeset says a run that fails while instrumenting or building its sandbox now reports the `instrument` phase as entered, but no scenario drove that failure. The new scenario writes projects whose `buildCommand` exits 1 and runs dry-run-only mutation runs in them. Through `Engine.mutationTestCell` and the stage event queue it asserts the run fails in the `instrument` stage with reason `Sandbox initialization failed`, after `prepare` then `instrument` were reported entered. Through `Engine.strykerCell` and the span exporter, in a second fresh project, it asserts the same stage failure and that the failed gathering and sandbox work stay inside the one `instrument` phase span. Against main's cells the phases read `['prepare']` and no phase span exists; on this branch both hold. Every scenario runs inside its temp project's working directory, so config discovery cannot pick up the package's own Stryker config.
ryanleecode
approved these changes
Oct 13, 2026
ryanleecode
left a comment
Contributor
There was a problem hiding this comment.
Root verdict on 235059e: round 1 was full depth with the seven enrolled packs. Rounds 2-3 were delta passes plus a full-PR MiniMax M3.1 cross-model pass. Round 4 was an exact-head correctness lens (MiniMax M3.1) over the conflict-free merge of main 449e9cd, which touches none of U1's files. Every finding is fixed in 63d0a04 or rejected by the validator with the reason kept. CI is green on the head, with 0 threads, and there's no overlap with main since its base. The plan is the accepted T10 plan with my gate changes. U1 moves the instrument mark, so R18 can re-read the instrument share for the T9 gate.
systemfsoftware-maker
added a commit
that referenced
this pull request
Oct 13, 2026
…t, wire-text stream decode Takes main through b08379d (#333, #343, #339, #320, #346, #344, #286, #348, and the gate per-project thresholds fix). Conflicts: - stryker-outputs.ts: main's labeled output list (#320) keeps this branch's two extra exclusions, each with its own label for the --ignorePatterns help: the incremental file's temp siblings (the killed-run fix) and an fs verdict store's directory. An output may now name several files. - instrument.cell.ts, plan-instrument.cell.ts: main's instrument-phase placement (#348), carrying this branch's fileContentDigests. - plan-request.cell.ts: main's planStageCell; the costs and probe schemas this branch retired stay gone, and the plan reads the one incremental file. - program-digest.integration.test.ts: this branch's config/program digest scenarios plus main's importerCheck scenario. - e2e-harness.fixture.ts, run-artifacts.fixture.ts: main's RunEventWireText decode (#339); the store-entry count stays and the checkpoint reader, with no caller left, does not come back. #286 added a local configKeyOf in ts-compiler.handle.ts, so this branch's schema export of that name becomes checkerConfigKeyOf.
This was referenced Oct 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
U1 of the T10 infection-filtered execution plan (
docs/plans/2026-10-13-1536-perf-infection-filtered-execution-plan.md, landed in this PR): R18, KTD9.The
instrumentphase mark (phaseEntered('instrument')) and theinstrumentphase span ran in the instrument cells' write phase, afterinstrumentFilesandmakeSandboxhad already run in the read phase. Instrumentation and the sandbox copy counted underprepare, and every Bench sample readinstrumentas 0-1 ms (A/A dispatch 38375500452). The instrument share that gates the T9 native-core restart could not be read.Both cells (
run/instrument.cell.tsfor runs,run/plan-instrument.cell.tsfor plans) now open the phase span and emit the mark at the start of the read phase. The write phase only offers file skips. No verdict, event shape or schema changes;phaseDurations.instrumentgrows andprepareshrinks by the same amount. Patch changeset for@systemfsoftware/stryker-js.Evidence
packages/stryker-js/tests/instrument-phase-span.integration.test.ts(oneFeature,.liveover real dry-run-only runs; OTelInMemorySpanExporter):Engine.strykerCellrun and anEngine.planRequestplan on a real temp project. The gather span (and, for the run,stryker.sandbox.acquire) lies inside the singleinstrumentspan, and the phase-mark span ends before gathering starts.buildCommandexits 1. TheEngine.mutationTestCellrun on the stage event queue fails withStageError { stage: 'instrument', reason: 'Sandbox initialization failed' }afterPhaseEnteredpreparetheninstrument. A secondEngine.strykerCellrun on a fresh project fails the same way, and its failed gather and sandbox work stay inside the oneinstrumentspan.main's cells, checked in a scratch worktree atorigin/mainwith this test file copied in: all 3 scenarios fail (insidePhase: [],markEndsBeforeGather: false, phases['prepare'],phaseSpans: 0). Green on this head.63d0a04ed,pnpm --filter @systemfsoftware/stryker-js exec vitest runexited 0 (full package suite, at1ecca4e2f; the only later change is this test file, rerun green), plustypecheck,dprint check,oxlint,lint:tsgo(0 errors) andpnpm lint:conventions. Head235059e45is a conflict-free merge oforigin/main(449e9cd53) into63d0a04ed, and the PR's five-file diff is unchanged by it. On that head: build, typecheck,format:checkand the three affected integration suites (15 tests). No mutation run anywhere (dry-run-only).instrumentat 0-0.5 ms on every leg because of the misplaced mark this PR fixes; see the plan's T9 section for the per-leg table. The T9 decision readsS=instrument / totalas the median of the first four green main push runs on or after this PR's merge commit. Skipped share is not applicable to U1.Code review
Four
ce-code-reviewrounds (mode:agent), the last on the exact head235059e45. Every finding below is fixed in a commit on this head or rejected by an independent validator with the reason kept.20261013-174110-ae429c85, basea83d1b32e: correctness, project-standards, testing, adversarial (in-process) and learnings with the seven enrolled packs. One P3, fixed (below). Plan fix: a stale "ci(bench)!: take the bench off the pull-request path #316 is open" blocker line was removed.03e7e4233..HEADplus a full-PR cross-model pass, runs20261013-190823-3280b24d,20261013-192400-eed6df5aand20261013-193111-5d19efcd(final, on63d0a04ed). In-process: correctness and testing on the delta, both with zero findings on the final head. Cross-model adversarial peer, served model MiniMax-M3.1-Flash-Preview (providerminimax-code, attested from the peer transcript's assistantmessage_endrecords). Final-head response ids: delta071dba1bcf195fd40401afcb797b90d2, full071dba7cbf2d5af419298f831c30866f. Read-only toolsread,grep,glob, no extensions.63d0a04ed..235059e45(the conflict-free merge oforigin/main449e9cd53, bringing in build(deps): bump pnpm-release-management from21fe619tob9b4a1d#344 and fix(stryker-js)!: stryker gate judges each project of a merged report against its own thresholds.break #299; 16 files, none of them U1's), run20261013-195252-fe602168. Served model MiniMax-M3.1-Flash-Preview (providerminimax-code, attested from the peer transcript's assistantmessage_endrecords, final response id071dbf84c8cce7b5dcabac3a3c3221bf). Read-only toolsread,grep,glob, no extensions. Zero findings. The reviewer found no merged-in file that touches U1's files or references the instrument phase,PhaseClockor the instrument span. Its residual notes (the#299gate'sthresholds.breakdecoding default, and journeys for--update-budget-baselineand files outside every listed project) concern main's already-merged gate code, not this PR, and are not dispositioned here.63d0a04ed: the failing-build scenario above63d0a04ed: the span run assertsphaseSpans: 1plus containment63d0a04ed: the span run writes its own fresh project63d0a04ed: the span run asserts the sameStageErrorstage and reasonchdirin the harnessforks+isolate: truegives each file its own process, scenarios run sequentially, and 13 sibling suites use the same pattern.strykerCellresolvesbasePathfrom cwd (RunEnvironment.service.ts:84)dryRunOnlynever reaches mutation testing, so no per-mutant events can fill the 256-slot queue; a hang still fails ontestTimeoutphaseDurations(raised twice)instrumentboundary thatphaseDurationsreports is themarkAtinside thestryker.phase.enteredspan (RunEnvironment.service.ts:132-140), so "mark span ends before gather starts" pins it. The repeat on the final head is suppressed under that verdict (unchanged code)fileCountcomes from the project map, commandfileCountfrom the read listProjectFiles.readAllis 1:1 (project-files.service.ts:84-93), so the counts are equal by construction; the attribute is telemetry-onlyinstrumentmarkSandwichrunsreadexactly once per cell run with no retry loop, every caller runs each cell once per run, and the old write-phase placement had the same multiplicityEvidence limits (not findings): reviewers suggested proving events and spans from one execution. That was tried this round and fails: an OTel layer on the
mutationTestCellstage path exports no spans, so the span half runs throughstrykerCellon its own fresh project. A configuredbuildCommandor node_modules symlink counts towardphaseDurations.instrumentbecausemakeSandboxruns inside the phase. Whether the Bench corpus configs set either is [unverified]; R18 readings should note it. Instrumenter-parse failures (Instrumenter failed) take the sameenteringInstrumentPhasepath but have no scenario of their own.Cross-stream notes
stryker/verdict-store) and ci(mutation): share each project's verdict store across shards and runs #289 addfileContentDigeststo the same two cells' read bodies. Root ordered U1 as its own PR offmain; whichever lands second resolves a small textual conflict.packages/stryker-js-instrumenterandpackages/stryker-js-plugin-interfaceare absent frommutation.ymlPROJECTS(:36); Stream K owns that file (CONST-E7). Not touched here; matters for U2/U3.