Skip to content

CI: PR cargo-test never executes crates/perry integration suites; main-push full run starves under merge trains (documented near-miss) #5960

Closed

Description

@proggeramlug

Documented near-miss from 2026-07-04: PR CI's change-scoped cargo-test (scripts/ci_test_scope.py) runs unit tests only — Running unittests src/lib.rs lines exclusively in the job log. The crates/perry/tests/*.rs integration/e2e suites are compiled (they appear in the changed-file scope listing) but never executed on the PR path. They only run on the full scope (push to main / --full).

Consequences observed today:

  • PR fix(hir): capture-snapshot P0 pair — resolved-name re-registration + assignment tracking #5938 landed with its own acceptance suite (capture_rereg_renamed_class.rs) failing — same_class_name_across_factories_keeps_snapshots_separate was red at the very commit that introduced it, through green required checks. (Fix in flight: the static-call/static-field lowering arms bypass class_renames.)
  • The intended safety net — the full run on main push — was cancelled repeatedly by the per-branch concurrency group during a rapid merge train (three squash-merges within ~20 minutes each superseding the previous run), so main went ~a workday with no completed full cargo-test.

This is a design trade (e2e suites compile binaries and take tens of minutes; scoping keeps PR latency sane), not a bug per se. Suggested cheap hardenings, any subset:

  1. Merge-train discipline (process, zero infra): after an admin merge, let the main-push full run complete before the next merge lands — or re-dispatch it after the last merge of a batch. (I'm adopting this immediately for my own merges.)
  2. Concurrency carve-out: give the main-branch full run cancel-in-progress: false (queue instead of cancel) so merge trains can't starve it.
  3. Scoped e2e tier: when the changed files include crates/perry/tests/<suite>.rs, run those suites (they're the PR's own acceptance tests — the exact class of miss that happened here).
  4. A nightly full run as a backstop (if not already present).

Refs: #5938 (the near-miss PR), the fix PR for the renamed-class static dispatch, run 28698034143 (an old-commit re-run that was the only completed "full" signal today, itself red on the known load-sensitive byob stall).

Activity

  1. proggeramlug commented on Jul 4, 2026

    @proggeramlug
    ContributorAuthor

    Correction to the premise after reading test.yml's trigger block: direct pushes to main deliberately do NOT trigger Tests (gates are PR + tags), and a nightly full-workspace cargo-test already exists (04:00 UTC cron) — suggestion 4 is moot and 'main-push full run' language above is wrong. The sharpened version of this issue is just two points: (1) the scoped PR path never executes crates/perry/tests integration suites — including a PR's OWN new acceptance suite, which is how #5938 landed with its acceptance test red; suggestion 3 (run the e2e suites named in the diff) is the cheap targeted fix. (2) Between a merge and the next nightly there is a window (up to ~28h) where a landed integration-suite regression is invisible; the nightly is the backstop, so protecting it from concurrency cancellation and treating its failures as first-class alerts matters. Today's window closes at tonight's nightly, which will be the first full cargo-test over v0.5.1230-1232 + the two fleet merges.

  2. added a commit that references this issue on Jul 5, 2026
  3. added 2 commits that reference this issue on Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions