Skip to content

Scope the de-skew's accuracy claim where the column outlives a DST transition - #3231

Merged
erikdarlingdata merged 1 commit into
devfrom
fix/deskew-dst-period-scope
Sep 9, 2026
Merged

Scope the de-skew's accuracy claim where the column outlives a DST transition#3231
erikdarlingdata merged 1 commit into
devfrom
fix/deskew-dst-period-scope

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

server_properties.utc_offset_minutes is DATEDIFF(MINUTE, GETUTCDATE(), GETDATE()) — the offset in force at collection time, one current value. Subtracting it from a stored timestamp is exact only while the timestamp and the offset sit on the same side of a DST transition. #3212 shipped sixteen de-skewed fields without saying so on the two reads where that can be false.

Raised by another session reading the merged code, and confirmed here against DarlingServerConnector.cs:94 and the fleet's reported offset.

Why get_index_usage is the case that matters

Thirteen of the sixteen describe current state — a running job, an open transaction, a cleaner that ran seconds ago — so their timestamps and the collected offset are always in the same DST period, and the de-skew is exact.

last_user_access is not current state. It is the GREATEST of four sys.dm_db_index_usage_stats columns that persist since the instance restarted, which on a stable production box is routinely months. So a large share of those values predate the most recent transition and come back 60 minutes early — silently, and in the plausible direction, which is the bad one. sqlserver_start_time on the same row is the bound on how far back it can reach.

The fleet is exposed rather than theoretically exposed: any target in a DST-observing zone has it — the offset differs by an hour between summer and winter, so every pre-transition timestamp is de-skewed by an offset that was not in force when it was written. A target configured to UTC is unaffected, and on AWS RDS the instance takes its time zone from a creation-time parameter, so a non-UTC zone is an ordinary configuration rather than an exotic one. #2932 records the measurement behind the four-hour figure above.

get_pvs_stats gets the same note one tier down: the off-row cleaner runs continuously and is unaffected in practice, but aborted_version_cleaner_* can reach back weeks on a quiet database — a live read showed 2026-08-26 against a 2026-09-09 snapshot — so it can cross a transition.

Documented rather than fixed, deliberately

The proper fix is a zone instead of an offset: CURRENT_TIMEZONE_ID() (SQL Server 2019+) collected alongside utc_offset_minutes, then AT TIME ZONE at the read boundary, which handles transitions correctly for a timestamp of any age. That is a collected-column addition, a migration rung, and swapping the - make_interval(...) expression for an AT TIME ZONE conversion in five readers, with the offset kept as the fallback while server_timezone_id is null — which it will be for every server until its next server_properties collection.

There is no backfill, and an earlier revision of this description wrongly said there was. Nothing stored needs rewriting: the label is DeSkewedAtRead precisely because the collector ships us.last_user_seek verbatim and the offset is subtracted in the projection, so no stored column is wrong today. And a single current zone ID is legitimately valid retroactively where a single current offset is not — that asymmetry is the whole defect. utc_offset_minutes changes twice a year, so today's value is wrong for half of history; CURRENT_TIMEZONE_ID() returns a zone identity that is stable for the life of the server, so collecting it going forward and applying it to already-stored rows is sound. It is the one field whose present value describes the past.

Two things that stay genuinely open, so this does not read as free: CURRENT_TIMEZONE_ID() is SQL Server 2019+, so the offset path has to be kept rather than replaced — the product supports servers older than that and will for someone regardless of any particular deployment — and a server whose Windows time-zone data disagrees with the historical rule for a given date is still wrong for that date. Much narrower than the current failure, not zero.

What this carries is the #2993 shape: the scope of the claim, stated where the de-skew happens, so the next reader knows the frame is exact inside the current DST period and within an hour outside it. Same family as #2993 by a different mechanism — there a zone was captured and discarded, here an offset is captured and over-applied, and both end with a naive timestamp confidently placed in the wrong frame.

Verification

Doc-only; no shipped SQL or behaviour changes. Both affected projects build with 0 errors and both files keep CRLF with no bare LF.

One check worth naming because it is a live trap in this area: ConsumedTimestampFrameDisciplineTests.ProjectionAlias is IgnoreCase over " as <word>" and reads string literals and comments, so ordinary prose next to a census column name registers as a SQL projection alias — it cost #3212 two CI rounds. I ran that exact regex over every added line and no added prose produces a reportable match.

Supersedes #3229, which is closed. Same content, opened fresh as a single commit so the branch history carries only the generalised form.

…ansition

server_properties.utc_offset_minutes is DATEDIFF(MINUTE, GETUTCDATE(), GETDATE())
-- the offset in force at COLLECTION time, one current value. Subtracting it
from a stored timestamp is exact only while both sit on the same side of a DST
transition, and #3212 shipped sixteen de-skewed fields without saying so on the
two reads where that can be false.

Thirteen of the sixteen describe current state -- a running job, an open
transaction, a cleaner that ran seconds ago -- so their timestamps and that
offset are always in the same period. get_index_usage is the exception and the
worst case: sys.dm_db_index_usage_stats persists since the instance restarted,
which on a stable production box is routinely months, so a large share of
last_user_access values predate the most recent transition and come back 60
minutes early. Silently, and in the plausible direction.

The exposure is general rather than particular: any target in a DST-observing
zone has it and a UTC-configured target does not, and on AWS RDS the instance
takes its time zone from a creation-time parameter, so a non-UTC zone is
ordinary configuration rather than an exotic setting -- which also kills the
plausible-sounding assumption that an RDS instance is probably UTC. #2932
carries the measurement behind the four-hour figure. sqlserver_start_time on the
same row bounds how far back the affected values reach.

get_pvs_stats is a tier less severe and gets the same note: the off-row cleaner
runs continuously, but aborted_version_cleaner_* can reach back weeks on a quiet
database, so it can cross a transition.

Documented rather than fixed, deliberately. The proper fix is a ZONE instead of
an offset -- CURRENT_TIMEZONE_ID() collected alongside utc_offset_minutes, then
AT TIME ZONE at the read boundary, which handles transitions for a timestamp of
any age -- and that is a collected column plus a migration rung, which is its
own change. There is no backfill: the collector ships the DMV value verbatim and
the offset is subtracted in the projection, so nothing stored is wrong today.
The offset path stays as the fallback regardless, because CURRENT_TIMEZONE_ID()
is SQL Server 2019+ and the product supports older servers.

What this carries is the #2993 shape: the scope of the claim, stated where the
de-skew happens, so the next reader knows the frame is exact inside the current
DST period and within an hour outside it.
@claude

claude Bot commented Sep 9, 2026

Copy link
Copy Markdown

Reviewed. This is a doc-only change (XML doc comments added to DarlingObjectStatsReader.cs and DarlingPvsReader.cs), no SQL or behavior changes — matches the PR description. The new prose is accurate against the code (utc_offset_minutes is indeed a single collection-time offset, last_user_access is a GREATEST over columns that persist since restart, and the PVS cleaner times can reach back weeks), and CRLF line endings are preserved in both files.

Lite/Darling parity gap: Lite has the exact same latent limitation in its counterpart readers, but this PR doesn't add the equivalent note there:

  • Lite/Mcp/McpObjectStatsTools.csget_index_usage de-skews last_user_access with r.LastUserAccess?.AddMinutes(-utcOffsetMinutes), where utcOffsetMinutes comes from McpServerLocalWindow.OffsetForAsyncGetServerUtcOffsetMinutesAsync, i.e. the same single current-value server_properties.utc_offset_minutes.
  • Lite/Mcp/McpPvsTools.csget_pvs_stats de-skews all four ADR cleaner timestamps the same way, including aborted_version_cleaner_start_time/end_time, which is the exact field this PR calls out as reaching back weeks on a quiet database.

Since this PR's whole point is "document the DST-period scope of the claim," and the two apps intentionally share this de-skew pattern (Lite is even called "Darling's twin" in the surrounding doc comments), the same caveat probably belongs in the Lite XML doc comments too — otherwise a future reader of the Lite side has no record of the same boundary. Left as inline comments on the two Lite files with the specific spots.

No correctness, security, or performance concerns beyond that — nothing here changes runtime behavior.

@erikdarlingdata
erikdarlingdata merged commit 5b6ad87 into dev Sep 9, 2026
8 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/deskew-dst-period-scope branch September 9, 2026 19:14
erikdarlingdata added a commit that referenced this pull request Sep 9, 2026
#3230 was open when this batch was cut and merged at 19:31:55Z while it was
being verified, making it the newest merge in the range. origin/dev is
merged in rather than rebased, so e459539 stays intact in the history.

Its body carries no entry text and neither queue directory holds any, so
the entry is written here from the description. No issue exists behind it,
so it cites its own number, and its definition takes its ascending place
between [#3226] and [#3231].
erikdarlingdata added a commit that referenced this pull request Sep 9, 2026
…d pull requests, and strike an unsound claim from #3199's (#3232)

* Record the CHANGELOG entries for seventeen changes across ten merged pull requests

Applies the [Unreleased] entries for the pull requests merged to dev after
#3213's batch pass, and strikes an unsound arithmetic claim from the #3199
entry that pass shipped.

CHANGELOG.md only: 17 entries prepended inside [Unreleased] -- 5 under
Added, 12 under Fixed -- plus the 15 link-reference definitions they need,
and one edited line. Entry text comes from each pull request's own body
where it carried one.

* Cite #3224's own number on its entry, not the pull request it follows

The Azure credential-recording entry led with [#3218], the pull request it
is a follow-up to, which resolves to a different change. No issue exists
behind #3224, so under the file's rule -- issue numbers where an issue
exists, pull-request numbers where one does not -- its own number belongs
in the citation position, with the follow-up relationship kept as prose.

#3224 was the only one of the ten merged pull requests with no issue behind
it whose number went uncited. The [#3218] definition is swapped for
[#3224] rather than added, since nothing else cited it.

* Absorb #3230, which merged mid-batch

#3230 was open when this batch was cut and merged at 19:31:55Z while it was
being verified, making it the newest merge in the range. origin/dev is
merged in rather than rebased, so e459539 stays intact in the history.

Its body carries no entry text and neither queue directory holds any, so
the entry is written here from the description. No issue exists behind it,
so it cites its own number, and its definition takes its ascending place
between [#3226] and [#3231].

* Take #3230's entry from its own CHANGELOG block, not its measurement prose

The entry was written from #3230's description before that description
carried an entry-ready block, and the description's measurement section was
taken at c9f04f3 -- before #3227's rebase moved Deadlock.cs
LastTranStartedLocal onto FormatServerClock, which took it out of both the
inventory and the string-literal subset.

KnownTruncatedRanges has 30 entries; the retired entry said 31, and carried
13, 544, 510, 2,238 and 42,927 besides. The 544 and 510 predate the
content-trim and cannot be recomputed from shipped code at all, and the file
and declaration totals move with every commit. The lane's own block names
only what shipped, so it replaces the entry wholesale rather than the
numbers being patched.

Its one quantity is cross-checked against the shipped array by the
verification battery, so the entry cannot restate a count the code does not
have.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant