Skip to content

State the refresh slot envelope against a measured maximum, and say a daily store-metrics point is a last snapshot - #3141

Merged
erikdarlingdata merged 1 commit into
devfrom
fix/3119-refresh-envelope-and-last-snapshot-caveat
Sep 7, 2026
Merged

erikdarlingdata merged 1 commit into
devfrom
fix/3119-refresh-envelope-and-last-snapshot-caveat

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

HeaviestHourlyRefreshObservedCeilingSeconds' envelope read its 306 s of clearance as the operating margin — "better than a third of the slot". That clearance is a property of the closed 16-run population the constant is the maximum of, not of current load. A census read of one closed day (2026-09-06, 22 runs, one row per run from timescaledb_information.job_history) puts the maximum at 778.4 s: 121.6 s of the 900 s slot, 13.5% of it, and 28.4 s past the 750 s watch line, which ClassifyRefreshSlotHeadroom bands ApproachingSlot.

The constant does not move, and that is a decision rather than an omission

594 is defined as the maximum of a published population and is pinned as exactly that — Verify and TheDerivedFiguresAreExactlyStateable_AndTheConstantIsThePopulationMaximum both require Ceiling == population.Max(). Raising it means republishing the population, and the census read that produced 778.4 s publishes a maximum and a run count, not 22 durations; a constant set from a figure whose population is not in the comment is the value-you-have-to-trust shape #3101 removed. Two arithmetic consequences say the same thing from the other end: the constant is an int and 778.4 is not one, and Ceiling * 10 % 60 == 0 is required because CompressionPhaseMinutes quotes the ceiling in minutes to one decimal — 778 fails it.

So what changed is the envelope stated against the constant, and the margin sentence is now attributed to the population it belongs to. The new paragraph states the measured envelope separately, with its own closed window.

TimescaleSupportTests' two assertions that red at a ceiling past 750 are untripped, because the ceiling did not move — and the ordering 480 < 594 < 750 that #3107 rests the rejected-alternative argument on is likewise untouched.

What the pin holds

The measured maximum is a quoted reading, on the same footing as the inadmissible snapshot readings; every figure the prose draws against it is derived and pinned in RefreshCeilingProvenancePinTests:

  • the slot margin, against RefreshPhaseSlotSeconds less the maximum;
  • the distance past the watch line, against RefreshSlotWarningSeconds — which also carries the strict-past claim, since no figure the pattern can hold equals a zero or negative difference;
  • the share of the slot, against the margin over the slot, truncated to one decimal as the prose states it;
  • the BAND, from the shipped ClassifyRefreshSlotHeadroom rather than from the two inequalities — placed ABOVE the arithmetic, because Verify is fail-fast and a maximum restated low enough to change its band trips the margin equality on the way past. A verdict clause underneath that one could not go red on its own, which is the property the rejected-alternative pin's own comment already names;
  • the CLOSING SCOPE, the same rule the snapshot enumeration carries: an absolute day that has ended, not a scope read against a clock a doc comment does not have.

EveryNumericPin_ReportsAnInjectedDrift sweeps all eight captured digits one at a time, so each of the four figures is individually load-bearing.

The surface half

get_store_metrics' daily series is DISTINCT ON (object_kind, object_name, date_trunc('day', metric_time)) ... ORDER BY ... metric_time DESC — the day's LAST snapshot. For 2026-09-06 that is 705.2 s, so the surface understates the day by 73.2 s and places it below the warning line when a run was above it. Anyone re-deriving this envelope from the obvious surface concludes nothing entered the band, and nothing on the surface said so. Added to the tool's description:

Each daily point is that day's LAST snapshot, never its maximum or its mean — the settled figure a growth question wants, but it means a MAXIMUM question (what was this job's longest run that day, did it enter its warning band) cannot be answered from this series: the day's peak is DROPPED rather than smoothed, so a day whose worst run crossed a threshold reads as a day that never approached it. The route that carries one row per run is TimescaleDB's own job history (timescaledb_information.job_history). The hourly snapshot behind the series has the same limit one grain down: it samples last_run_duration once an hour at a fixed offset, so a run longer than that offset is never recorded at all.

TheDescription_SaysADailyPointIsALastSnapshotAndNotAMaximum asserts that sentence and the shipped SQL TOGETHER, so either half moving lands there: a read that became max-preserving would make the caveat wrong in the other direction, and the pairing is what routes a reader to the description instead of only telling them the query changed.

The same wrong pointer was in two operator-facing places and both are corrected rather than left for a follow-up. HeaviestRefreshRuntimeSql' doc called collect.store_metrics "the complete per-run series"; it is the same hourly grain taken by a different sweep, and the daily point served from it is the day's last. The ApproachingSlot log line offered that series as the per-run history an operator should go read; it now names timescaledb_information.job_history first and says what the hourly series is, pinned in TheRefreshSlotLogLine_IsLeveledByBand_AndSaysNothingWithoutAReading.

The open-ended claims that were left

Two more in the same file, both about this envelope, both now closed to the set they were measured over. RefreshSlotWarningSeconds said 750 s is 26% above "the largest run the narrowed regime has been observed to produce" and that a reading in the band is "this job doing something its own record has no instance of" — the record now has one. It states the 26% against the closed population's maximum and points at the live envelope rather than restating its figures, so they keep one home. The five-sixths paragraph said a line at 83.3% "leaves the observed peak a full sixth of the slot below it and cannot cry wolf on load this store has already carried"; it now says that of the five readings it names.

#3137's NoOpenPopulationClaim_SurvivesOutsideTheSentenceThatRejectsIt stays green, and one mutation re-proves it against prose this change adds rather than against the three sites it was written for.

Mutations, each in isolation

Verified on disk before running, restored after, on a committed tree.

mutation red
maximum restated 700.4 s (in-slot, out of band) the VERDICT clause: bands 700.4 s as InsideSlot rather than ApproachingSlot
maximum bumped to 778.9 s stated slot margin 121.6 s against 121.1 s derived
margin restated 194.8 s (the last-snapshot answer) stated slot margin 194.8 s against 121.6 s derived
share restated 21.6% stated 21.6% of the slot against 13.5% derived
distance past the line restated 24.8 s stated distance past the watch line 24.8 s against 28.4 s derived
closing scope replaced by "the most recent day read" no longer CLOSED by an absolute day
the whole figure clause deleted PIN PARSE MISS, whose message offers update / correct / delete
an open population claim added to the new prose #3137's guard: 1 doc comment(s) ... quantify over a population of readings without closing it
the description's caveat sentence deleted pattern 'daily point is that day's LAST snapshot[^.]*maximum' did not match
the description's per-run route deleted 'timescaledb_information.job_history' not in given string
the log line's route reverted to collect.store_metrics 'timescaledb_information.job_history' not in given string
the daily SQL's tiebreak flipped to metric_time ASC the description pin's SQL half, beside the pre-existing ORDER BY pin

Local verification

Darling.Tests targets net10.0-windows, so the real test sources were compiled into a throwaway net10.0 console harness behind a minimal xUnit shim, OUTSIDE the worktree: RefreshCeilingProvenancePinTests, DarlingMcpStoreMetricsToolsTests, DocCommentHygieneTests, FleetIdentifierScrubTests and TimescaleContinuousAggregateTests whole, plus eight [Fact] bodies copied verbatim out of TimescaleSupportTests. 86 tests, 86 passed, 0 Error(s) from the same build. EveryCrefTargetResolvesOrIsBounded and the per-solution-project cref floors pass, so the new crefs resolve and the whole-tree sweep read the whole tree.

What is not here

The census read has been taken for one day. The ceiling's own "ONE READING IS LEFT OPEN" paragraph and the sample-versus-census limitation on the days after the boundary are unchanged, because a single day's census discharges neither. OtherHourlyRefreshObservedCeilingSeconds' 140 s is still un-re-derived for the same reason its doc already gives.

CHANGELOG entry text

  • The refresh slot envelope is stated against a measured maximum, and get_store_metrics says a daily point is a last snapshot ([The hourly refresh envelope is stated at 306 s of slot margin; a measured run leaves 121.6 s and sits inside the warning band, and the daily series structurally cannot show it #3119]) - HeaviestHourlyRefreshObservedCeilingSeconds read its 306 s of clearance as the operating margin. That clearance belongs to the closed 16-run population the constant is the maximum of; a census read of one closed day over timescaledb_information.job_history puts the maximum at 778.4 s, which leaves 121.6 s of the 900 s slot (13.5% of it) and sits 28.4 s past the 750 s watch line - a reading ClassifyRefreshSlotHeadroom bands ApproachingSlot. The constant does not move: 594 is pinned as the maximum of the population the comment publishes, and the census read publishes a maximum and a run count rather than 22 durations, so raising it would replace a re-derivable value with one a reader has to trust. What moved is the envelope stated against it, and the margin sentence is now attributed to the population it describes. Every figure drawn against the measured maximum is derived and pinned - the slot margin, the share, the distance past the line, and the band taken from the shipped classifier rather than restated, placed above the arithmetic because Verify is fail-fast and a verdict clause underneath the margin equality could not go red on its own - plus a closing-scope check that requires an absolute day rather than a scope read against a clock a doc comment does not have. The durable half is the surface. get_store_metrics' daily series is a DISTINCT ON ... metric_time DESC selection, so each point is that day's LAST snapshot: for the measured day it reads 705.2 s, understating the day by 73.2 s and placing it below the warning line when a run was above it, which means anyone re-deriving the envelope from the obvious surface concludes nothing entered the band. The description now says a daily point is a last snapshot rather than a maximum or a mean, that a maximum question cannot be answered from the series, and that the per-run route is timescaledb_information.job_history - asserted together with the shipped SQL, so a read that became max-preserving lands on the same pin instead of leaving the caveat wrong in the other direction. The two operator-facing places that offered the hourly self-metrics series as a per-run history - HeaviestRefreshRuntimeSql' doc and the ApproachingSlot log line - name the per-run route instead. Two remaining open-ended claims about this envelope are closed to the sets they were measured over: the watch line's "26% above the largest run the narrowed regime has been observed to produce", whose record now has an instance in the band, and the five-sixths choice's "cannot cry wolf on load this store has already carried". Darling

…y store-metrics point is a last snapshot

The envelope on HeaviestHourlyRefreshObservedCeilingSeconds read its 306 s of
clearance as the operating margin. That clearance belongs to the closed
population the constant is the maximum of; a census read of one closed day puts
the live maximum inside the warning band, leaving 121.6 s. The constant stays at
594, which is still the maximum of the population it names.

get_store_metrics' daily series is a last-snapshot-per-day selection, so a
maximum question asked of it is answered with the day's final reading. The
description now says so and names the per-run route.
@claude

claude Bot commented Sep 7, 2026

Copy link
Copy Markdown

Reviewed. This PR is doc-comment / MCP tool description / log-message text only — no SQL or control-flow changes (HeaviestHourlyRefreshObservedCeilingSeconds stays at 594, ClassifyRefreshSlotHeadroom's thresholds are untouched).

Checked and confirmed consistent:

  • New live-envelope arithmetic in the doc comment (778.4s measured max → 121.6s slot margin, 13.5% of slot, 28.4s past the 750s watch line) is internally correct against the 900s slot and 750s watch line, and matches the new RefreshCeilingProvenancePinTests derivation exactly (verified the tenths-based integer math by hand).
  • The new regex/pin in RefreshCeilingProvenancePinTests correctly accounts for DocProseFor's existing <b>-stripping normalization, so it matches the <b>778.4 s</b>-style markup actually in the source.
  • No stale references to the old prose ("better than a third of the slot", "collect.store_metrics" described as the "complete per-run series", the old log-line wording) remain anywhere else in the test suite.
  • All new <see cref> targets (ClassifyRefreshSlotHeadroom, RefreshSlotHeadroom.ApproachingSlot, RefreshSlotWarningSeconds) resolve within the same file.
  • This feature (TimescaleDB job-history / store self-metrics) has no Lite counterpart — Lite uses DuckDB and doesn't run TimescaleDB background jobs — so there's no parity drift to flag here; Darling-only is correct.
  • No T-SQL, no new SQL surface, no input handling/secrets/file/network/process changes — nothing in scope for the security or T-SQL-style checks.

No issues found.

@erikdarlingdata
erikdarlingdata merged commit 2c9943a into dev Sep 7, 2026
8 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/3119-refresh-envelope-and-last-snapshot-caveat branch September 7, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant