Repository navigation
Stop query_stats from recording a false zero CPU when only the CPU delta is unknowable (#4394) - #4423
Merged
Merged
Conversation
…lta is unknowable Refs #4394.
erikdarlingdata
marked this pull request as ready for review
September 26, 2026 14:14
erikdarlingdata
added a commit
that referenced
this pull request
Sep 26, 2026
The MCP raw top-N reads apply the hourly rollups' zero-interval filter, so raw and hourly agree over the same window. TopQueriesSql, TopQueriesByHostObjectSql and TopProceduresSql in DarlingDataReader take TimescaleSupport.IntervalHonestSourceFilter, the filter the interval-honest rollups already use. A zero-interval row carries zero deltas (#4423 now records a real interval when execution or elapsed time is knowable), so no figure changes in real data. Tests: a SQL-shape pin, and TopQueriesHourlyRoutingLiveTests extended with a zero-interval row, where raw and hourly totals now match through GetTopQueriesByCpuRoutedAsync. Refs #4394
This was referenced Sep 26, 2026
erikdarlingdata
added a commit
that referenced
this pull request
Sep 26, 2026
…ts (#4431) When a cached plan's counters restart, query_stats now decides the whole row at once, instead of each counter on its own. - ICollectorDeltaCalculator.DecideRow is new. Its default implementation is a no-op, so existing implementers keep compiling. It peeks at every counter family under the key without updating anything, and reports whether any family would reset. - CollectorDeltaCalculator.DecideRow is the shared core both hosts run. Any decrease makes the whole row a reset. The #2235 series-age test decides what the row becomes. If the restart falls inside the gap since the previous pass, every counter reads as its current value over the real interval. Otherwise the whole row is unknown, (0, 0). - QueryStatsCollector.WritePayload calls DecideRow once per row, before the eight per-family delta calls, which still update their own baselines. It then overrides all eight deltas and the interval inputs when the row restarted. A row with no reset is unaffected. The ResolveWorkerDelta fallback from #4423 still applies afterward. - It applies to query_stats only. - Tests: QueryStatsRowCoherentResetTests goes through the real WritePayload and calculator. It covers two field-shaped restarts, a row with no reset, an unchanged CPU over a real interval, and a restart the age test can't place. Refs #4428
This was referenced Sep 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #4394.
Why
QueryStatsCollector.WritePayloadmakes eight independent delta calls per row, one per counter family, and keeps only the worker (CPU) call's interval assample_interval_seconds. When the worker family's delta call takes the no-delta path (a first sighting, a counter reset, or a gap past the policy) it reports interval 0 — the honest signal for "this counter's delta is not knowable." But the exec-count and elapsed-time families are keyed and evaluated independently, and in the field they sometimes DO have a real, knowable delta in the same pass where CPU does not. The row was written with CPU 0 andsample_interval_seconds0 regardless, and the interval-honest filter (sample_interval_seconds IS DISTINCT FROM 0) then discarded that row's real executions and elapsed duration along with the fabricated CPU zero.Observed on two production SQL Server stores over 24 hours: 183 and 8,844 rows respectively (up to 0.43% of fleet-wide elapsed time), up to 100% of a rare query's day on one store, one row that carried up to 50 seconds of elapsed time, and on one store 86% of the affected rows showing zero reads with a long tail of nonzero ones.
What changes
execIntervalSeconds,elapsedIntervalSeconds) instead of discarding them without _.QueryStatsCollector.ResolveWorkerDelta, decides what to write for CPU and the row'ssample_interval_seconds:delta_worker_timeas NULL and takessample_interval_secondsfrom the exec interval (falling back to elapsed if exec is also 0).Not changed
WorkerZeroOverARealInterval_KeepsTheMeasuredZero).query_stats_interval_hourly) still filters onsample_interval_seconds IS DISTINCT FROM 0, not ondelta_worker_time, so a NULL-CPU row with a real interval is included in the rollup's executions/elapsed sums and simply skipped bysum()/min()/max()for the worker column — no continuous-aggregate change was needed or made.ICollectorRowWriter.Value(long?)already exists on both the PostgreSQL (Darling) and DuckDB (Lite) writers, anddelta_worker_timeis already nullable in both stores, so this needed no migration.Test plan
Darling/Darling.Tests/QueryStatsUnknownCpuTests.cs): five cases againstResolveWorkerDelta— worker unknown + exec known, worker+exec unknown + elapsed known, the stakeholder's real-zero case, all-unknown, and a normal delta. RED on dev: the helper doesn't exist there, so this is a compile-time RED (confirmed on a detached worktree at dev's tip, 15 compile errors, allCS0117/CS8130pointing at the missing member).[Fact]): drives a realCollectorDeltaCalculatorwhere the exec and elapsed families already know the row's key from a prior pass but the worker family sees it for the first time, and a recordingICollectorRowWriter. Assertsdelta_worker_timeis NULL andsample_interval_secondscomes from the exec interval. Included in the same RED-on-dev compile failure above.Darling/Darling.Tests/QueryStatsUnknownCpuLiveTests.cs, own scratch database per#1776): afterPgMigrations.MigrateAsync, plants one normal row and one NULL-CPU row (real interval, real executions/elapsed) for the same query hash, refreshesquery_stats_interval_hourly, and asserts the hourly rollup's execution-count and elapsed sums include both rows while the worker sum equals the normal row alone. A second assertion runs the Viewer'sworker_time_per_secondexpression (CAST(delta_worker_time AS double precision) / NULLIF(sample_interval_seconds, 0) / 1000.0) directly against both rows: it returns a number for the normal row and NULL — not an error — for the unknown one.timescale/timescaledb:2.30.1-pg18container (removed after the run): all new tests green, plus the existingDeltaSeriesAgeTestsclass (25 total, 0 failures).Darling.TestsandLite.Testson macOS withEnableWindowsTargeting=true: 0 warnings/errors on both. Did not find an existingCollectorPayloadArityTests/QueryStatsCollectorDefinitionTestsassertion that pinsdelta_worker_time's nullability or arity in a way this breaks;Lite.Testsbuilt clean unmodified.Darling.Tests/Lite.Testssuites end to end given the time remaining in this pass — only the touched/related classes above. CI will run the full suites on Windows.CHANGELOG entry
SECTION: Fixed
ENTRY:
query_statsrow whose CPU (worker time) counter reset or was seen for the first time, while its execution-count or elapsed-time counters had a real delta in the same pass, previously wrote CPU as a false 0 over a fabricated zero interval — which then caused the row's real executions and duration to be dropped by the interval-honest filter along with the fake CPU. That row now records CPU as unknown (NULL) and keeps its real interval, executions, and duration. A CPU counter that is genuinely unchanged over a real measured interval still records 0, unchanged.REF:
[#4423]: #4423