Skip to content

CI: per-test Lite timings as an artifact, and temporary cap guards while #5208 is open - #5211

Merged
erikdarlingdata merged 2 commits into
devfrom
ci/5208-lite-timing-and-caps
Oct 5, 2026
Merged

erikdarlingdata merged 2 commits into
devfrom
ci/5208-lite-timing-and-caps

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Part of #5208. Lite.Tests on the hosted Windows runners has grown from about 9½ minutes of test time to about 45 in five days (+7 min/day, with only ~110 tests added). Last night's nightly was cancelled at its 90-minute build cap during the unsharded Lite run, so no release was published. This PR doesn't fix the slowdown. It makes the slowdown measurable, and keeps CI and the nightly from failing on it while it's found.

What changes

  • Per-test timings: every Lite.Tests run in CI now writes an xUnit v3 result file (-xml) with per-test durations, uploaded as an artifact kept for 7 days:
  • A summary in the log: .github/scripts/lite-timing-top.py prints the 25 slowest classes, by summed test time, and the 25 slowest tests, right after each run.
    • The step and the upload are if: always() and continue-on-error: true, so they can never turn a job red.
    • A missing file only means a missing artifact.
  • Two temporary guards, each commented as a guard against timeout reds, not a fix, to be brought back down when Lite.Tests on hosted Windows runners grew from ~9½ to ~45 minutes in five days; the unsharded nightly run now exceeds its 90-minute cap #5208 is fixed:
    • lite-tests: timeout-minutes 20 → 30. The slowest shard reached 19 minutes on Oct 5.
    • The nightly build job: 90 → 150. Its Lite step took 45, then 64, then more than 89 minutes on three consecutive nights.
    • The release path: its job has no timeout-minutes (the 360-minute default), so it's unchanged.

Checked

  • The -xml <file> option: confirmed on xunit.v3 4.0.1's in-process runner on Windows 11. A one-class run wrote a schema-version-3 results file.
  • The summarizer was checked against a sample results file.
  • The workflow-text pins pass on Windows: CiShardCommandLineBudgetTests (the per-chunk command-line budget), NightlyVersionInjectionTests, LockedModeRestoreCoverageTests, DarlingPathFilterGateTests and the other classes that read these files. Darling 205/0, Lite 79/0.
  • Both workflows parse (actionlint and PyYAML).

SECTION: None (CI only)

@erikdarlingdata
erikdarlingdata marked this pull request as ready for review October 5, 2026 07:59
@erikdarlingdata erikdarlingdata added the merge-train In a merge train: its CI ran on the train PR label Oct 5, 2026
@erikdarlingdata
erikdarlingdata merged commit ec03d8a into dev Oct 5, 2026
20 of 29 checks passed
@erikdarlingdata
erikdarlingdata deleted the ci/5208-lite-timing-and-caps branch October 5, 2026 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-train In a merge train: its CI ran on the train PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant