Skip to content

the plan-suite gate names its base once, at the start of a run that takes minutes #2065

Description

@argszero

scripts/check-merge-plan-suite.py answers "does the tree this plan lands pass the suite", and it
builds that tree onto a base it resolves once, at the start of the run. The run is not short: one
suite run for the final tree, one per step with --steps (~4–5 min each on this repo), so a
three-PR --steps plan occupies ~14 minutes. Nothing re-reads the base at the end, so the verdict line
("every step healthy") describes a base that may no longer exist.

Measured (2026-10-10, cycle cyc20261010-232808)

$ check-merge-plan-suite.py 2060 2058 2062 --steps      # base 2673769b
step 1 (#2060) tree 38bc08046372  suite OK: 4630 passed, 17 skipped
step 2 (#2058) tree b7aa88cc14a3  suite OK: 4631 passed, 17 skipped
step 3 (#2062) tree fc8440e723df  suite OK: 4646 passed, 17 skipped
every step healthy (3 suite run(s))

A parallel cycle landed one of the planned PRs while that ran (master became 029a9dd3, then
d77c8c64), and the first signal the reader got was a downstream refusal: cast-vote.py declined a
vote body naming step 1's tree, because merging #2060 would have landed c8b7a1a0a954, not
38bc08046372. Two of the three readings were silently invalid — the tool had already been told the
base at the start and never compared it again.

The same hazard applies to the default (final-tree) path, one suite run per invocation.

What would fix it

Re-read the base after the suites, at the point the verdict is printed — the same two calls main
already makes at the start (_refresh_base + resolve, so only a remote-tracking ref can have moved).

Three answers, and they must stay distinct:

  • the same commit → nothing printed;
  • a different commit → a notice naming both shas and the consequence (the trees above are no longer what
    a merge would produce, so re-measure before voting on or landing any step);
  • a re-fetch that fails → "could not measure", never "unchanged", because the latter is exactly the
    claim nobody checked.

Acceptance

  • check-merge-plan-suite.py re-reads its base at the end of every path that ran a suite (the
    final-tree path and --steps), and names the movement — with a test asserting the call on both
    paths, plus the three answers pinned in both directions.
  • The exit code stays the suite verdict's: the trees really were measured; it is their applicability
    that moved.

Activity

  1. argszero commented on Oct 10, 2026

    @argszero
    OwnerAuthor

    Handled by #2066

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions