CI: replace the 39 tests that read the workflow files with actionlint and one release-safety script #5947

Claude posting for Erik Darling.

39 test files read the CI workflow and script files as text and assert on what they say. Since 2026-09-15, half of the commits that changed CI also had to edit one of these tests. This issue proposes to replace them with actionlint and one small check script, and to keep only the checks that protect a release.

Source: git log on dev since 2026-09-15, and grep over the test projects at dev 36159005e.

What it costs (measured)

• Size of the CI code: the workflows and scripts under .github/ are 19,018 lines.
  • build.yml is 3,267 lines, and 1,147 of them are comments.
  • nightly.yml is 1,700 lines, and ci-select.py is 2,039.
• Tests that read it: 39 files in Darling.Tests and Lite.Tests mention .github/, build.yml or nightly.yml. The largest by mention count are Lite.Tests/CrossAppGuardCiGateTests.cs, Lite.Tests/CiSelectMapSelectGuardTests.cs, Darling.Tests/DarlingPathFilterGateTests.cs, GuardStageWorkflowTests.cs, LockedModeRestoreCoverageTests.cs and CiSelectReplayGuardTests.cs.
• The edit cost:
  • 89 commits changed .github/workflows or .github/scripts since 2026-09-15.
  • 43 of those 89 also edited one of the 39 test files.
  • The 39 files were edited in 82 commits in total.
• The run cost: build.yml is on the keep-full list (ci-select.py:693), as build configuration. So every workflow change already runs the full suite once. With the tests that read the workflow, a wrong guess about a pinned line costs a second full run.

Proposal

1. Add a fast job that runs actionlint when a file under .github/workflows/ changes. It finds YAML, expression and shell mistakes that the text tests do not look for.
2. Move the checks that protect a release into one script under .github/scripts/, run by that job. Examples are the publish order after the signature checks (Release: the Linux tarball and container image publish before the Windows signature guard has passed #5804) and the Guard stage running before the shards. A script can read the YAML as data, not as text.
3. Delete the other tests that assert on the text of CI files. Apply the rule from Tests that assert spelling, not behavior: cap them, stop the review asking for pins, delete the shape ones #5833: a test that fails on a correct change protects nothing.
4. Keep ci-select.py's own unit tests of its logic, such as replay and map selection, where they test behavior with inputs, not the text of build.yml.
5. For new CI machinery, the PR states the time it saves, and a follow-up measures it. The selection, shard and map code already spans thousands of lines, and each part carries tests of its own.

Risk

• A text test can guard a rule that nobody wrote down. Before deleting a test, name the defect it fails on. If there is one, move the check into the script.

Done when

• actionlint runs on workflow changes.
• One script holds the release-safety checks of the CI files.
• No test in Darling.Tests or Lite.Tests asserts on the text of a file under .github/workflows/.