You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Darling: the Query Store read's check for rows with no start time reads every raw row in the window #4972
Darling's Query Store table read first checks whether the server's window still holds raw rows with no interval start time. Only a store upgraded in the last few days has them. That check is an EXISTS over collect.query_store_stats with interval_start_time_utc IS NULL, bounded to the server and the window.
On a field store the answer is almost always "none". To say so, the read fetches every heap page in the server's window, because the only index it can use is on (server_id, collection_time). On a large store that was 855,000 raw rows and 56,000 blocks for a 24-hour window.
#4969 first tried a partial btree on (server_id, collection_time) WHERE interval_start_time_utc IS NULL. On a field store the index is empty, so the check reads almost nothing. It came out of #4969 because nothing builds it safely while the service runs:
collect.query_store_stats is a hypertable, and TimescaleDB refuses CREATE INDEX CONCURRENTLY on a hypertable.
The per-chunk form, WITH (timescaledb.transaction_per_chunk), takes a ShareLock on each chunk while it builds it. The notes in PgTableTuning say it also takes a ShareLock on the hypertable first, which holds up writes to every chunk. A lock sampling for Darling builds a background index so get_query_store_top no longer reads every entry a server holds (#4952) #4969 saw no lock on the hypertable, so settle which is right. A cancel partway through leaves the hypertable's index invalid.
The collector's write has a 10-second deadline. A lock held longer than that fails the write.
No "newest chunk only" rule makes this safe. The Query Store backfill writes rows with a backdated collection_time (the end of the slice it fills), so it writes into older chunks too (QueryStoreBackfill.cs).
Options
Add the index to the PgTableTuning startup list, the path that already builds hypertable indexes when the service starts. Measure how long the build takes on a field-sized store, and compare it with SetupTimeoutSeconds.
Build the index with CONCURRENTLY on each chunk table directly, since a chunk is a plain table. Index each new chunk as it is created. Check how TimescaleDB treats chunk indexes that the hypertable does not define, for compression and for new chunks.
Hold the collector's and the backfill's writes to a chunk while its index builds. Or give those writes a longer deadline during a build.
Keep a small table of which servers and time ranges still hold raw rows with no start time. The collector writes it when it stores such a row, and the check reads it instead of the raw table.
Measure first
Measure how much of the read's time the check takes on a field store, at 24 hours and at 168 hours, before choosing.
Fixed in #5021, merged to dev as acf6c10. Darling now builds a partial index on collect.query_store_stats (server_id, collection_time) WHERE interval_start_time_utc IS NULL at service start, before collectors run. The legacy-row check reads that near-empty index for the uncompressed chunks, so a 24-hour window's check no longer reads recent raw rows. Compressed chunks are still read by their columnar scan, so a 168-hour window keeps that part of the cost. The check's SQL is unchanged and stays exact.
Activity
Record the production scan and check numbers for the legacy-row index (…
Darling builds a partial index at start so the Query Store legacy-row…
erikdarlingdata commentedon Oct 3, 2026
Fixed in #5021, merged to dev as acf6c10. Darling now builds a partial index on
collect.query_store_stats (server_id, collection_time) WHERE interval_start_time_utc IS NULLat service start, before collectors run. The legacy-row check reads that near-empty index for the uncompressed chunks, so a 24-hour window's check no longer reads recent raw rows. Compressed chunks are still read by their columnar scan, so a 168-hour window keeps that part of the cost. The check's SQL is unchanged and stays exact.