Repository navigation
Carry an operator's postgresql.conf lines below the include across a major upgrade (#4358) - #4405
Merged
Merged
Conversation
…on step for #4358 ManagedConfMigration.ExtractOperatorLinesBelowInclude(postgresqlConf) returns every raw line strictly after the darling-managed.conf include, in original order, using the same PostgreSQL-accurate include detection Rewrite and HasManagedInclude already use. A Legacy conf (no working include) returns nothing -- #4358 is scoped to the migrated shape. Unit pins (ManagedConfExtractOperatorLinesTests.cs, no live PG needed): - a line below the include is returned; - a line above the include never appears in the extracted set; - no include returns empty; - a differently-pathed but equivalent include is still recognized (parser, not raw string match); - a conf ending exactly at the include returns empty, not a stray blank. Refs #4358.
Alongside CarryAutoConfAsync, post-swap (one pattern, per the 07:17Z ruling). Reads the OLD (retained) cluster's postgresql.conf, extracts operator lines below the darling-managed.conf include via ManagedConfMigration.ExtractOperatorLinesBelowInclude, and probes each assignment line one at a time against the new binaries with postgres -C, the same isolation CarryAutoConfAsync gives auto.conf settings. A comment, blank line, or operator include directive is carried unconditionally (never probed -- there is no setting name to ask -C about). A rejected line is logged and skipped, never fatal. Carried lines are appended AFTER the legacy blocks context.AppendManagedConf already wrote, under ManagedConfMigration.MovedOperatorLinesComment (reused per the ruling), so an operator's line still wins. Never throws: an unexpected failure resets postgresql.conf to its pre-carry baseline and logs a warning naming the retained old data directory as the manual fallback. Build: Darling.Tests 0 warnings / 0 errors on macOS (EnableWindowsTargeting=true). Pins (extending ManagedConfUpgradePathTests etc. for the wiring order and the live _Gated upgrade-path pin) are still open -- see the PR body / handoff note for what's left. Refs #4358.
…e-never-fatal, and a live upgrade-path pin
erikdarlingdata
marked this pull request as ready for review
September 26, 2026 12:12
This was referenced Sep 26, 2026
Closed
erikdarlingdata
added a commit
that referenced
this pull request
Sep 26, 2026
…ow live (#4446) The upgrade-path tests now read the preload line where the managed settings live: darling-managed.conf, not postgresql.conf. - A new shared helper, ManagedPreloadAssert, reads both postgresql.conf and darling-managed.conf. It asserts all of these: - exactly one effective shared_preload_libraries line across the two files; - that line is in the managed file, and the include is present; - it names both timescaledb and pg_stat_statements. - RuntimeAdvance_SameMajorStoreWithRealData_SwapsWithoutPgUpgradeAndKeepsEverything_Gated (both cases) uses it. It also reads pg_settings' setting and sourcefile on a non-pooled connection, and expects both libraries, sourced from darling-managed.conf. - UpgradeInPlace_OperatorLineBelowInclude_CarriedAndApplied_NoManagedKeyDuplicated_Gated counts the preload line across both files, and keeps its pg_file_settings applied check. - ManagedConfFileTests.RenderBody_PreloadLine_IsExactlyOne_WithBothLibraries pins the render itself: exactly one preload line, with both libraries. - Test-only. Refs #4336, #4405, #4358
This was referenced Sep 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #4358.
Why
DarlingStoreUpgrade.CarryAutoConfAsynccarries onlypostgresql.auto.confacross a major PostgreSQLupgrade. Lines an operator adds below the
darling-managed.confinclude inpostgresql.confwere notcopied into the new cluster; they stayed behind in the old (retained) data directory. #4336's docs point
operators at
ALTER SYSTEMas the only carried path today; #4358 closes that gap forpostgresql.confitself.
What changes
ManagedConfMigration.ExtractOperatorLinesBelowInclude(string postgresqlConf)(pure, no disk I/O):given a
postgresql.conf's text, returns every raw line strictly after thedarling-managed.confinclude, in original order, using the same PostgreSQL-accurate include detection
Rewrite/HasManagedIncludealready use (never a raw string match). A legacy conf (no working include)returns nothing — this change is scoped to the migrated shape.
DarlingStoreUpgrade.CarryOperatorConfLinesAsync: wired intoUpgradeDataDirectoryAsyncimmediately after
CarryAutoConfAsync(post-swap, one pattern). Reads the OLD (retained) cluster'spostgresql.conf, extracts the below-include region, and probes each ASSIGNMENT line one at a timeagainst the new binaries with
postgres -C— the same per-setting isolationCarryAutoConfAsyncgives auto.conf settings. A comment, blank line, or operator
include/include_if_exists/include_dirdirective is carried unconditionally (there is no GUC nameto hand
-C). A rejected line is logged (name only; the same secret-name-fragment withholdingCarryAutoConfAsyncalready applies) and skipped — never fatal to the upgrade. Carried lines areappended to the new cluster's
postgresql.confAFTER the legacy blockscontext.AppendManagedConfalready wrote (step
"conf-new-cluster"), underManagedConfMigration.MovedOperatorLinesComment(reused — provenance need not distinguish a same-major move from a cross-major carry). Never throws: an
unexpected exception resets
postgresql.confto its pre-carry (legacy-appended) baseline and logs awarning naming the retained old data directory as the manual fallback, matching
CarryAutoConfAsync'sown "never brick a completed upgrade" posture.
Test plan
ManagedConfExtractOperatorLinesTests.cs(5 unit[Fact]s, no live PG):Total: 5, Failed: 0.DarlingStoreUpgradeTests.CarryOperatorConfLinesAsync_CarriesAfterTheLegacyBlocks_OperatorValueWins(ordering pin): after the carry, the new cluster's
postgresql.confholds the carried operator blockAFTER the legacy block, parsed with
ParseConfText, and the operator'slog_min_duration_statementvalue wins over the legacy block's own copy of the same key.
DarlingStoreUpgradeTests.CarryOperatorConfLinesAsync_RejectedLine_IsSkippedAndLogged_OthersStillCarried_NeverFatal(rejected-line-never-fatal pin, via the substitutable probe delegate — no real postgres.exe): one
candidate line the probe rejects is skipped and logged (
NOT carried: darling_4358_unknown_setting),the other two candidates are still carried, and the method returns normally.
Both new pins run as
dotnet Darling.Tests.dll -class Darling.Tests.DarlingStoreUpgradeTests -method "*CarryOperatorConfLinesAsync*"→Total: 2, Errors: 0, Failed: 0.d20aa68e4(before theCarryOperatorConfLinesAsyncwiring commit): copying the newtest file into a detached worktree at that commit and building fails to COMPILE —
CS1061: 'DarlingStoreUpgrade' does not contain a definition for 'CarryOperatorConfLinesAsync'— themember does not exist yet, so both new unit pins are RED by compile failure, as expected.
_Gated— extendsManagedConfUpgradePathTests.cs—UpgradeInPlace_OperatorLineBelowInclude_CarriedAndApplied_NoManagedKeyDuplicated_Gated: runs a realend-to-end major upgrade (
DarlingManagedPostgres.EnsureRunningAsync) with an operator line(
log_min_duration_statement = 4358) appended below the include in the OLD cluster's migratedpostgresql.conf. Asserts the line is present in the NEW cluster'spostgresql.confafter the upgrade,APPLIED per
pg_file_settings(notpg_settings.sourcefile, which still names postgresql.conf for anunchanged value), and that
shared_preload_libraries = 'timescaledb'appears exactly once (no managed keyduplicated between the legacy block and the carried operator block). Builds clean (0 Warning(s), 0
Error(s), both
Darling.Tests.csprojandLite.Tests.csproj), and is correctly SKIPPED on this macOS run(1 Skipped in the
ManagedConfUpgradePathTestsclass run) because it needsDARLING_TEST_PGRUNTIME_OLD/DARLING_TEST_PGRUNTIME_NEWZIPand the managed runtime is Windows-only. On the pre-Carry operator lines below the darling-managed.conf include across a major PostgreSQL upgrade #4358 code theoperator line assertion fails outright:
DarlingStoreUpgradehas noCarryOperatorConfLinesAsyncandUpgradeDataDirectoryAsyncnever calls it, so the operator's below-include line never reaches the newcluster at all — both the presence assertion and the
pg_file_settings.appliedassertion fail. CI decidesthis pin on Windows with the PG runtime fixtures set.
ManagedConfMigrationStateTests(Total: 6, Failed: 0),ManagedConfRewriteOperatorSafetyTests(Total: 5, Failed: 0),ManagedConfUpgradePathTests(
Total: 5, Failed: 0, Skipped: 1— the new_Gatedpin).Darling.Tests.csproj -p:EnableWindowsTargeting=trueandLite.Tests.csproj -p:EnableWindowsTargeting=true— 0 Warning(s), 0 Error(s) on macOS, after every change in this session.CHANGELOG entry
SECTION: Fixed
ENTRY:
postgresql.conflines below thedarling-managed.confinclude across a major PostgreSQL upgrade ([Carry an operator's postgresql.conf lines below the include across a major upgrade (#4358) #4405]) - previously onlypostgresql.auto.conf(ALTER SYSTEM) settings survived a major upgrade; a hand-edited line below the include stayed behind in the retained old data directory. Each carried line is now probed against the new binaries and skipped (never fatal) if rejected.REF:
[Carry an operator's postgresql.conf lines below the include across a major upgrade (#4358) #4405]: Carry an operator's postgresql.conf lines below the include across a major upgrade (#4358) #4405