Skip to content

Plan regression analysis falls back to the raw query_store_stats read on almost half its passes #5513

Closed

Description

@erikdarlingdata

Claude posting for Erik Darling.

Found by the #5508 audit. The numbers are from its step 1 (comment 6051350950): pg_stat_statements over 13.2 days on a store with 43 servers at schema V165. Line numbers are at dev 25f68ae.

What runs

CollectPlanRegressionFactsAsync (Darling/PerformanceMonitor.Darling.Analysis/PgFactCollector.QueryPerf.cs, about lines 660 to 720) picks one of two reads for the PLAN_REGRESSION fact.

  • When QueryStoreIntervalLatest.ReadsTableAsync (Darling/PerformanceMonitor.Darling.Storage/QueryStoreIntervalLatest.cs:408) says the interval table covers the window, it reads the table.
  • Otherwise it reads raw collect.query_store_stats with PlanRegressionSql (QueryPerf.cs:312).

Numbers

  • The raw read ran 12,510 times, under two statement ids with the same text.
  • It read 605 M blocks and ran for 99,159 s, about 27.5 hours. That is 47,586 and 51,341 blocks per call.
  • The table read ran 14,866 times. So about 46% of passes took the raw path.

Why it matters

#5507 and the V168 per-day totals make the table path cheaper. Neither touches the raw path.

Not yet known

Why ReadsTableAsync returns false this often. Candidates are coverage that starts after the window's start, a pending coverage row, or a server that was never backfilled. This is inferred from the code, not observed.

Fix direction

Find the cause first, with a read-only query on the coverage rows. Then fill the coverage, or let the table serve the days it covers and read raw rows only for the rest. Code only. No schema rung is expected.

Activity

  1. added
    in-progressActively being worked by a local session or its agents (PR open or in flight)
    on Oct 8, 2026
  2. erikdarlingdata commented on Oct 8, 2026

    @erikdarlingdata
    OwnerAuthor

    Claude posting for Erik Darling.

    Closing this with the big store's evidence (#5508 comment 6053411189, section 1). The raw reads no longer run.

    • All 43 servers now read the table. Every server has a coverage row and no pending batch.
    • pg_stat_statements recorded no new call of the two raw reads from 03:00 to 05:56 UTC on 2026-10-08. The table read kept running.
    • The coverage claims all began on 2026-09-26 between 04:19 and 04:46 UTC. Until raw's floor passed that time, each pass read raw. Raw keeps about 4 days, so that lasted until about 2026-09-30. The 46% came from those days, because the statistics were last reset on 2026-09-24.

    A server added later reads raw until raw's floor passes the start of its own coverage, about 4 days. That fallback is by design.

    PR #5531 is closed unmerged. The review found a gap in the table rule that is older than that PR, filed as #5541.

  3. removed
    in-progressActively being worked by a local session or its agents (PR open or in flight)
    on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions