Repository navigation
Decide a re-created statement entry's counters row-coherently in pg_statement_stats (#4428) - #4435
Merged
Merged
Conversation
Remove the committed intended-failure mutation fact; the mutation instead gets exercised as an uncommitted, restored code change to CollectorDeltaCalculator.DecideRow, confirmed RED then GREEN.
erikdarlingdata
marked this pull request as ready for review
September 26, 2026 16:54
erikdarlingdata
deleted the
fix/4428-pg-statement-stats-row-coherent
branch
September 26, 2026 16:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #4428.
Why
On the PostgreSQL-target store, about 2,600
pg_statement_statsrows a day (of 4.65M) carry a zero interval alongside a nonzero delta, scattered through ordinary passes. That is a statement entry deallocated and re-entered under the same key (queryid, dbid, userid, toplevel), with some counters re-grown past their old values before the next look — the same mechanism thequery_statscase (#4431) already handles. These are not whole-pass resets: the only ≥90% passes are one first pass per server, with no deltas.Deciding each of the three delta families (
calls,total_exec_time,rows) independently means one family can read as a shrink (reported unknowable) while a sibling that had already re-grown past its own pre-restart value reads as an ordinary, wrongly inflated increase — the same row telling two different stories about whether it restarted.What changes
PgStatementStatsCollector.ReadAsyncnow decides the restart once per row, row-coherently, usingICollectorDeltaCalculator.DecideRow(added in #4431) before any of the three per-family delta calls mutate a baseline. If any family would decrease, the whole row is treated as a restart.Placement of that restart uses
stats_since(pg_stat_statements 1.11+, PostgreSQL 17) — the moment the entry was (re-)created in the extension's hashtable — compared only against the previous pass's value of the monitored server's ownnow(), never against the collector host's clock. Bothstats_sinceandnow()are read in the same query, as two new (unstored) columns appended after the existingstatements_stats_resetcolumn.stats_sincefollows the same version-then-relation-existence pattern already used fortoplevel, because it too is a column of the extension's own catalog version, not the engine major. On Aurora,aurora_stat_statements()carries everypg_stat_statementscolumn unguarded, so it reads the same waytoplevelalready does there.Where the target-clock pass window lives: rather than repurpose the calculator's per-key
(Value, Timestamp)baseline (which is stamped with the collector's clock) or add a second cache to the collector,ICollectorDeltaCalculatorgained one small additive member,PreviousPass(serverId, group, observedTime), which rolls and returns a caller-named pass window exactly like the private mechanismDecideRowalready uses internally, but keyed under a caller-chosen group name that never collides with an ordinarycollectorName. This lets the collector track the target's own clock's previous value without inventing a new cache shape or touching any other collector's behavior.If
stats_sinceis inside the gap since the previous pass's target-clocknow(), every counter's delta becomes its current value over the real interval (the restart's whole accrual). Ifstats_sinceis absent, or the previous target-clock pass is unknown (first sighting), orstats_sincefalls at or before the previous pass, the whole row is (0, 0) — never crediting a restart that cannot be honestly placed.The payload and schema are unchanged; the two new query columns are read but not stored.
Test plan
New file
Darling/Darling.Tests/PgStatementStatsRowCoherentResetTests.cs, five pins:stats_sinceinside the gap on the target clock) → every counter credited at its current value.stats_sincebefore the previous pass's targetnow()→ (0, 0).stats_sinceabsent → (0, 0).stats_sincereads "after" the previous pass on the collector's own clock but before it on the target's clock must not be credited — proving placement never touches the collector's clock.All five run green in-process on macOS:
Total: 5, Failed: 0.The mutation (
CollectorDeltaCalculator.DecideRowforced toreturn default;before its own body, so it never reports a row restart) was applied as a temporary, uncommitted code edit and confirmed RED, then reverted and confirmed GREEN again:The clock mutation checks that the restart is placed on the target's clock. The previous-pass time was anchored to the collector's
CollectionTimeinstead of the target'snow(), as a temporary, uncommitted edit.Reset_TargetClockSkew_NeverComparedAgainstCollectorClockthen fails withAssert.Equal() Failure: Expected: 0, Actual: 5(line 178): a pre-gap entry was credited. Restored, all 5 pass.Total: 5, Failed: 1—Reset_StatsSinceInsideGap_CreditsCurrentValuesfailed.Total: 5, Failed: 0.Runtime RED on pre-fix
origin/dev(e0232a498), not just a compile failure: a dev-compatible copy of pin 1's scenario (same two passes, same rows, calls 1 → 16 / total time 57,695,259 → 703,943, nostats_since/target_nowcolumns since dev'sRowlacks them) run against dev's ownPgStatementStatsCollector.ReadAsync, which decides each of the three delta families independently:Assert.Equal() Failure: Values differ — Expected: 16, Actual: 15(DeltaCalls; dev's ownPgStatementStatsCollector.ReadAsyncreads the shape as a mixed per-family result rather than crediting a coherent 16, confirming dev lacks the row-coherent decision this change adds — not merely a compile-time gap from the newPreviousPassmember).Also run and green:
QueryStatsRowCoherentResetTests(part of an 88/88 combined run alongsidePgStatementStatsRowCoherentResetTestsandDocCommentHygieneTests,Total: 88, Failed: 0),PgSchemaGeneratorTests(24/24),PgStatementTextTests(11/11),DeltaFamilyIntervalCompletionRungTests(9/9),PostgresFaultOutcomeTests(90/90),PostgresTargetConfigTests(41/41).DeltaFamilySeedingCensusTestslives inLite.Tests(build-only on macOS, cannot run in-process);Lite.TestsandDarling.Testsboth build 0 errors / 0 warnings-of-note (Darling.Tests carries pre-existing warnings unrelated to this change).CHANGELOG entry
SECTION: Fixed
ENTRY:
REF:
[#4435]: #4435