Repository navigation
Derive the planner's test-file inputs from the gate scripts that run them #4473
Description
Problem
Whether a scripts/*.test.mjs file runs, and whether a change to it schedules the lane that runs it, are two facts written in three places:
- the file itself;
- an npm gate script (
check:release,check:asf-source) or anode --teststep in.github/workflows/ci.yml; - a literal path list in
scripts/ci-test-plan.mjs(RELEASE_CONTRACT_FILES,ASF_SOURCE_FILES, and six siblings — 120 hand-written entries across eight sets).
Adding a test requires editing all three. Missing the third means the gate still runs the test, but no change to the test itself selects the lane that runs it, so the test can be edited green.
This has now happened at least three times:
- three of the release gate's own tests reached no lane;
RELEASE_CONTRACT_FILESnamed ascripts/prepare-windows-upgrade-baseline.test.mjsthat did not exist, for about a month;agent-run-store.test.tsstayed in the storage stress set for about a month after refactor(storage): make SQLite the sole operational authority #1994 deleted it.
Two guardrails in scripts/ci-test-plan.test.mjs catch the drift after the fact and both have fired — every test a gate script runs reaches a lane that runs that gate and the planner names no path that no longer exists. They convert a silent failure into a loud one, which is a real improvement over nothing, but they exist only to police a duplication that does not need to be there.
Suggested direction
Have the planner derive the test-file entries instead of listing them: read package.json, take every scripts/*.test.mjs a gate script names, and treat it as an input to that gate's lane. Three places become one, and both guardrails lose their reason to exist.
The trade-off worth deciding explicitly: this makes the planner's output depend on npm script text, where today it is a static table. The planner runs before npm ci, so reading package.json is free, but it does widen what a package.json edit can change.
.github/workflows/ci.yml names about eleven more test files directly in node --test steps rather than through an npm script. Those are already covered — the steps that run them are unconditional — but if the derivation is worth doing, it is worth deciding whether those should be derived too, or left alone because an unconditional step needs no selection at all.
Scope note
Split out of #4461, which narrowed several gates and made each one derive its inputs from its own authority. This is the same principle applied to a different list, but it changes planner behaviour for every lane rather than only the ones that PR narrowed, so it was left out deliberately rather than forgotten.
Not in scope
The two guardrails should stay until the derivation lands. They are what makes this issue's failure mode visible at all.
Activity
Rangsh commentedon Sep 1, 2026
take
Rangsh commentedon Sep 1, 2026
@Astro-Han Hi, this issue has been resolved. I have submitted PR #4479 . Could you please take a look when you have a chance?