Repository navigation
Don't re-hold the raw purge for an empty successor mid-upgrade (#4300) - #4406
Merged
Merged
Conversation
erikdarlingdata
marked this pull request as ready for review
September 26, 2026 11:39
This was referenced Sep 26, 2026
Closed
erikdarlingdata
added a commit
that referenced
this pull request
Sep 26, 2026
…et (#4443) Adds a live pin that the raw-purge seam probe stops at the successor's first bucket above the frozen legacy's end. - Upgrade_NonEmptySuccessor_HoleInItsOwnSpan_SeamProbeStopsAtItsFirstBucket_RawPurgeStaysCovered seeds three things: a frozen legacy, a non-empty successor with no seam hole, and one unmaterialized hour inside the successor's own span, above its first bucket. - IsRawTierDropSafeAsync must still read Covered. The seam probe covers only the gap between the legacy and the successor, not the successor's own span. - The existing tests stayed green when the probe's upper bound was widened to now() - 1 day. Under that change, this pin fails: the probe walks into the successor's span, finds the hole and reads Short. - Test-only. Refs #4406
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #4300.
Why
An upgrade with no outage should not re-hold the raw purge just because a
freshly-shipped successor rollup has not run its first refresh yet. The
purge-arming probe's stitched fallback (used when the successor holds no
bucket above the frozen legacy's last one) treated every hour between the
legacy's freeze and right now as an unprobed hole, reporting the coverage
slot as Short. On a healthy store mid-upgrade that's a false alarm: the
successor's own first refresh reaches every bucket newer than its refresh
start offset on its own, so those recent hours are never actually at risk
from the raw purge (which only drops chunks older than its own multi-day
retention window). The false alarm sent the operator toward a "hold" log
line for a state that clears itself within about the refresh's own start
offset.
What changes
RetentionArmSafetySql's legacy-branch fallback now ends the probe windowat
now() - HourlyRefreshStartOffsetinstead of a barenow(). The probe'sother bound (the successor's first bucket above the legacy's last one, when
one exists) is unchanged, as is the
- INTERVAL '1 hour'bucket-widthtrim. The fallback still derives entirely from the same
HourlyRefreshStartOffsetconstant already used everywhere else the hourlyrefresh window matters — no new literal was introduced. The class doc
comment's sentence about the upper bound is updated to describe the new
bound and why it's safe.
Test plan
dev, GREEN on this branch):TimescaleContinuousAggregateTests.RetentionArmSafetySql_StitchedSlot_FallbackUpperBoundUsesHourlyRefreshStartOffsetasserts the generated SQL carries the offset-adjusted fallback and no
longer carries the bare
now()form. RED ondev(fails: old SQL has nooffset subtraction), GREEN on this branch. Full class: 44/44 pass on this
branch.
dev, GREEN on this branch):FrozenRollupLiveTests.Upgrade_NoOutage_EmptySuccessor_RawPurgeStaysCoveredseeds a legacy frozen a few hours back, raw rows every hour up to now, and
an empty successor.
IsRawTierDropSafeAsyncmust readtrue. RED ondev(readsfalse), GREEN on this branch.devand this branch —the safety pin):
FrozenRollupLiveTests.Upgrade_RealOutageInsideTheFallbackWindow_RawPurgeStaysNotSafeseeds the legacy's last bucket 3 days back and raw rows between 3 and 1
days back that neither side ever materialized — a genuine hole sitting
entirely inside the new fallback's window.
IsRawTierDropSafeAsyncstaysfalseon both commits: the fix only narrows the fallback's upper bound,it never widens what counts as a hole below it.
FrozenRollupLiveTests21/21 pass,TimescaleContinuousAggregateTests44/44 pass,
MaterializationHoleRepairTests18/18 pass,RollupBackfillLiveTests9 pass / 3 skipped (Windows-only prerequisites,as before), 0 failed across all four classes.
Darling.TestsandLite.Testsboth build 0 errors / 0 warnings withEnableWindowsTargeting=true. Both targetnet10.0-windowsand cannotrun on this platform; the live results above come from running
Darling.Tests.dllin-process on this Mac against a real container, notfrom CI, which still decides the Windows-only classes.
CHANGELOG entry
SECTION: Fixed
ENTRY:
REF:
[Don't re-hold the raw purge for an empty successor mid-upgrade (#4300) #4406]: Don't re-hold the raw purge for an empty successor mid-upgrade (#4300) #4406