Skip to content

Decide a re-created statement entry's counters row-coherently in pg_statement_stats (#4428) - #4435

Merged
erikdarlingdata merged 3 commits into
devfrom
fix/4428-pg-statement-stats-row-coherent
Sep 26, 2026
Merged

erikdarlingdata merged 3 commits into
devfrom
fix/4428-pg-statement-stats-row-coherent

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

Refs #4428.

Why

On the PostgreSQL-target store, about 2,600 pg_statement_stats rows a day (of 4.65M) carry a zero interval alongside a nonzero delta, scattered through ordinary passes. That is a statement entry deallocated and re-entered under the same key (queryid, dbid, userid, toplevel), with some counters re-grown past their old values before the next look — the same mechanism the query_stats case (#4431) already handles. These are not whole-pass resets: the only ≥90% passes are one first pass per server, with no deltas.

Deciding each of the three delta families (calls, total_exec_time, rows) independently means one family can read as a shrink (reported unknowable) while a sibling that had already re-grown past its own pre-restart value reads as an ordinary, wrongly inflated increase — the same row telling two different stories about whether it restarted.

What changes

PgStatementStatsCollector.ReadAsync now decides the restart once per row, row-coherently, using ICollectorDeltaCalculator.DecideRow (added in #4431) before any of the three per-family delta calls mutate a baseline. If any family would decrease, the whole row is treated as a restart.

Placement of that restart uses stats_since (pg_stat_statements 1.11+, PostgreSQL 17) — the moment the entry was (re-)created in the extension's hashtable — compared only against the previous pass's value of the monitored server's own now(), never against the collector host's clock. Both stats_since and now() are read in the same query, as two new (unstored) columns appended after the existing statements_stats_reset column. stats_since follows the same version-then-relation-existence pattern already used for toplevel, because it too is a column of the extension's own catalog version, not the engine major. On Aurora, aurora_stat_statements() carries every pg_stat_statements column unguarded, so it reads the same way toplevel already does there.

Where the target-clock pass window lives: rather than repurpose the calculator's per-key (Value, Timestamp) baseline (which is stamped with the collector's clock) or add a second cache to the collector, ICollectorDeltaCalculator gained one small additive member, PreviousPass(serverId, group, observedTime), which rolls and returns a caller-named pass window exactly like the private mechanism DecideRow already uses internally, but keyed under a caller-chosen group name that never collides with an ordinary collectorName. This lets the collector track the target's own clock's previous value without inventing a new cache shape or touching any other collector's behavior.

If stats_since is inside the gap since the previous pass's target-clock now(), every counter's delta becomes its current value over the real interval (the restart's whole accrual). If stats_since is absent, or the previous target-clock pass is unknown (first sighting), or stats_since falls at or before the previous pass, the whole row is (0, 0) — never crediting a restart that cannot be honestly placed.

The payload and schema are unchanged; the two new query columns are read but not stored.

Test plan

New file Darling/Darling.Tests/PgStatementStatsRowCoherentResetTests.cs, five pins:

  1. Field-fixture shape (calls re-grown, total time dropped, stats_since inside the gap on the target clock) → every counter credited at its current value.
  2. stats_since before the previous pass's target now() → (0, 0).
  3. stats_since absent → (0, 0).
  4. Clock skew: target clock five minutes ahead of the collector's; an entry whose stats_since reads "after" the previous pass on the collector's own clock but before it on the target's clock must not be credited — proving placement never touches the collector's clock.
  5. No reset → unchanged, ordinary per-family deltas.

All five run green in-process on macOS: Total: 5, Failed: 0.

The mutation (CollectorDeltaCalculator.DecideRow forced to return default; before its own body, so it never reports a row restart) was applied as a temporary, uncommitted code edit and confirmed RED, then reverted and confirmed GREEN again:

The clock mutation checks that the restart is placed on the target's clock. The previous-pass time was anchored to the collector's CollectionTime instead of the target's now(), as a temporary, uncommitted edit. Reset_TargetClockSkew_NeverComparedAgainstCollectorClock then fails with Assert.Equal() Failure: Expected: 0, Actual: 5 (line 178): a pre-gap entry was credited. Restored, all 5 pass.

  • Mutated: Total: 5, Failed: 1 — Reset_StatsSinceInsideGap_CreditsCurrentValues failed.
  • Restored: Total: 5, Failed: 0.

Runtime RED on pre-fix origin/dev (e0232a498), not just a compile failure: a dev-compatible copy of pin 1's scenario (same two passes, same rows, calls 1 → 16 / total time 57,695,259 → 703,943, no stats_since/target_now columns since dev's Row lacks them) run against dev's own PgStatementStatsCollector.ReadAsync, which decides each of the three delta families independently:

  • Assert.Equal() Failure: Values differ — Expected: 16, Actual: 15 (DeltaCalls; dev's own PgStatementStatsCollector.ReadAsync reads the shape as a mixed per-family result rather than crediting a coherent 16, confirming dev lacks the row-coherent decision this change adds — not merely a compile-time gap from the new PreviousPass member).

Also run and green: QueryStatsRowCoherentResetTests (part of an 88/88 combined run alongside PgStatementStatsRowCoherentResetTests and DocCommentHygieneTests, Total: 88, Failed: 0), PgSchemaGeneratorTests (24/24), PgStatementTextTests (11/11), DeltaFamilyIntervalCompletionRungTests (9/9), PostgresFaultOutcomeTests (90/90), PostgresTargetConfigTests (41/41). DeltaFamilySeedingCensusTests lives in Lite.Tests (build-only on macOS, cannot run in-process); Lite.Tests and Darling.Tests both build 0 errors / 0 warnings-of-note (Darling.Tests carries pre-existing warnings unrelated to this change).

CHANGELOG entry

SECTION: Fixed
ENTRY:

REF:
[#4435]: #4435

Remove the committed intended-failure mutation fact; the mutation instead gets exercised as an uncommitted, restored code change to CollectorDeltaCalculator.DecideRow, confirmed RED then GREEN.
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 26, 2026 16:54
@erikdarlingdata
erikdarlingdata merged commit bf612de into dev Sep 26, 2026
15 of 16 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/4428-pg-statement-stats-row-coherent branch September 26, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant