CI: a quarantine list, so a test that breaks dev stops failing unrelated PRs while its fix is made #5942

Claude posting for Erik Darling.

When a test breaks on dev, every pull request that runs it fails until a fix PR merges. CI has no way to set a broken test aside. In the week of 2026-10-09 to 2026-10-16, this made 10 unrelated PRs fail. They lost about 20 PR-hours and ran CI 15 extra times.

Source: GitHub Actions data for build.yml, runs created 2026-10-09 00:00Z to 2026-10-16 11:50Z (1,126 runs), and the logs of the 275 failed test jobs. Code read at dev 67835f03d.

What happened (measured)

• 9 test classes went red on dev in the week, for 62 minutes to 33 hours each.
• 8 of 86 completed dev push runs ended red.
• Three of those breaks failed PRs that did not touch the broken test:

• Each of those PRs merged dev after the fix and ran CI again. Their failed runs used 873 runner-minutes.
• A small test-fix PR needs a median of 19 minutes for its CI run, and a median of 41 minutes from that run to merge. So a broken test blocks other PRs for about an hour at best.
• build.yml, .github/scripts/ci-select.py and the shard scripts have no quarantine or known-failing list. The Guard classes and the whole-tree guards job always run, so a broken Guard test fails every PR.

Proposal

1. Add one file, for example .github/test-quarantine.txt. Each line names one test or one class, with an issue number, the date it was added, and an expiry date at most 7 days later.
2. ci-select.py reads the file, and every test runner excludes the listed tests on pull requests and dev pushes. That covers the Guard job, the build job's test step, the Darling PostgreSQL shards, the Lite shards and the whole-tree guards.
3. The nightly still runs the listed tests. It reports their results in its summary, but they do not fail the nightly. So a quarantined test stays visible.
4. A Guard check fails when an entry is past its expiry date or names a test that does not exist. So the list cannot grow quietly.
5. A change to only this file must not force a full run. It must not be on the keep-full list.
6. The process: when a test breaks on dev, the first PR adds its line. The real fix follows in its own PR and removes the line.

Risks

• A real regression can hide behind an entry. The 7-day expiry and the nightly report limit how long.
• People can quarantine a test instead of fixing it. The expiry forces either a fix or a renewal with a reason.

Decision needed

• Can a Guard class be quarantined? This proposal says yes, with the same expiry. A broken Guard test fails every PR, so it is the case where quarantine helps most.

Done when

• The quarantine file exists, and all five runners named in item 2 honor it.
• The expiry check exists.
• The nightly summary lists the quarantined tests and their results.