Repository navigation
CI: run the store-upgrade tests on pull requests that touch the store's schema or upgrade path - #4500
Merged
Merged
Conversation
…'s schema or upgrade path The gated upgrade-in-place and same-major-swap E2Es (#1706, #3906) only ran on the nightly job: pull-request CI never built the upgrade fixture or set the three env vars those tests read, so they silently skipped on every PR. Adds a store-upgrade path filter derived from what those E2Es exercise (the store schema, the service project holding DarlingManagedPostgres and DarlingStoreUpgrade plus the managed-conf migration machinery, the two fixture scripts, and the workflow/filter files themselves). When a PR's changed files match it, build.yml's darling-pg job builds the same fixture nightly.yml builds and sets DARLING_TEST_PGRUNTIME_OLD/_NEWZIP/_PREVIOUS on the shards that carry the gated classes; other shards and non-matching PRs are unaffected. Pins the filter's existence and coverage in DarlingPathFilterGateTests. nightly.yml is unchanged.
The fixture steps and the three DARLING_TEST_PGRUNTIME_* env expressions were gated on matrix.shard != 0 because today's upgrade classes happen to hash into shards 1 and 2. That meant a renamed class, or a new upgrade E2E that hashes into shard 0, would silently skip the fixture with no signal. A class's shard must never decide whether it runs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Two gated end-to-end tests exist only to catch a store-upgrade regression before it ships: an
upgraded-in-place major upgrade over a real store with a hypertable, TOAST-sized plan XML, a
continuous aggregate and a compressed chunk, checksum-compared before and after; and a same-major
runtime swap with no
pg_upgradeinvolved. Both are gated on three environment variables(
DARLING_TEST_PGRUNTIME_OLD/_NEWZIP/_PREVIOUS) that point at a fixture nightly CI builds andsets every night — and only there. Pull-request CI never built that fixture and never set those
variables, so both tests silently skipped on every single pull request, including ones that changed
the exact code they exist to cover.
What changes
store-upgrade:path filter in.github/darling-paths-filter.yml, covering the storeschema, the service project holding the upgrade orchestration and the managed-conf migration
machinery it drives, the two fixture-build scripts, and the workflow/filter files themselves (the
same self-validating reasoning the existing
darling:filter already uses forbuild.yml).build.yml's PG test job now builds the same upgrade fixture nightly builds, and sets the samethree environment variables on the test step, on every shard, so a class's shard doesn't decide
whether it runs — only when a pull request's changed files match the new filter. Every PR that
doesn't touch this filter's paths is unaffected; no new step runs, so the build time for those legs
is unchanged.
nightly.ymlis untouched.DarlingPathFilterGateTestsasserting the filter exists and covers the store schemadirectory and both fixture scripts; a companion test asserts an injected removal of the schema entry
is detected. Both run green on this branch; the missing-entry case was verified RED by hand before
restoring the file (see below).
Test plan
dotnet build Darling/Darling.Tests/Darling.Tests.csproj -c Release -p:EnableWindowsTargeting=true:0 warnings, 0 errors.
python3 -c "import yaml; yaml.safe_load(...)"on both workflows and the filter file: clean.actionlintonbuild.yml: no new findings (the one pre-existing shellcheck note is innightly.yml, which this PR does not touch).DarlingPathFilterGateTestsrun in-process on the built assembly: 8/8 green, including the two newfacts.
Storageentry from the new filter's patternlist, rebuilt nothing (the test reads the YAML at run time), reran the same class — both new facts
failed as expected (
TheStoreUpgradeGate_CoversTheSchemaAndTheFixtureScriptsAndTheServiceProjectandTheStoreUpgradeGate_MissingTheStorageEntry_IsDetected), then restored the file and reran — 8/8 greenagain.
PgMigrations.cs: every PG shard ran the fixture steps (the cached download ~6–10 s, the previous-major runtime build ~26 s) and passed.[SKIP]lists hold none of the upgrade tests. Shard 0 skipped 6 (the live-SQL-Server and live-PostgreSQL-target E2Es, and a cert shape), shard 1 skipped 2 (the PG-log live tests), and shard 2 skipped 0.DarlingStoreUpgradeTests,DarlingManagedPostgresTestsandManagedConfUpgradePathTestsran their gated facts.pg_upgradecall site (theserverOptionsargument dropped; the argument-builder unit pins don't reach that call):DarlingStoreUpgradeTests.UpgradeInPlace_OldMajorStoreWithRealData_UpgradesAndKeepsEverything_Gated(shard 2) andManagedConfUpgradePathTests.UpgradeInPlace_OperatorLineBelowInclude_CarriedAndApplied_NoManagedKeyDuplicated_Gated(shard 1).devstill contains this branch's workflow and filter files, whichstore-upgrade:includes on purpose (so a change to the gate exercises the gate). Not for merge: CI proof (upgrade-touch-lite) #4498 therefore ran the fixture too, as designed.store-upgradedoesn't match, the new steps'if:is false and the env expressions give an empty string, which every gated test treats as unset (IsNullOrWhiteSpace→ skip).CHANGELOG
None: CI-only change.