Skip to content

Route the Query Store trend from cached rollup coverage instead of a fresh min/max(bucket) read - #4618

Merged
erikdarlingdata merged 7 commits into
devfrom
fix/4611-trend-bounds-cached
Sep 28, 2026
Merged

erikdarlingdata merged 7 commits into
devfrom
fix/4611-trend-bounds-cached

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 28, 2026 •

Copy link
Copy Markdown
Owner

Fixes #4611. Also carries #4605's rollup materialization ceiling (RollupCoverage.CeilingOf), which this uses.

Summary

get_query_store_duration_trend and the desktop Query Store trend chart used to run min(bucket)/max(bucket) over query_store_stats_corrected_hourly on every call, to decide where the rollup-served region ends and the raw tail begins. Both callers already probe and cache a RollupCoverage for the whole store on a timer, and that coverage now carries each rollup's measured materialization ceiling. This change reads the floor and ceiling for the corrected hourly rollup out of that cache instead of issuing a fresh bounds query on every trend load.

The ceiling (#4605)

RollupCoverage gains a per-view materialization CEILING: the instant at or above which a rollup holds nothing yet, read from the engine's own watermark (or a max(bucket) + bucket width fallback) in the same probe cycle that measures the floor, and cached the same way. CeilingOf answers null for a view that was never probed, is absent, or has materialized nothing — the safe "unknown" reading that keeps a caller on the raw path. Its own pins: an unknown-view pin, a cache-merge pin, and a live test against a real TimescaleDB (RollupCoverageCeilingLiveTests) proving the ceiling matches the engine's watermark, advances on a further refresh, and holds while the cache's reuse window is still open.

What changed

  • QueryStoreTrendRouting gets a ResolveAsync(RollupCoverage, NpgsqlDataSource, CancellationToken) overload. When the coverage carries a ceiling for query_store_stats_corrected_hourly, it builds the route directly from CeilingOf/FloorOf — no store round trip. CeilingOf is already the exact exclusive boundary the old pure decision computed as newest bucket + bucket width, so it becomes RawStartUtc with no adjustment.
  • When the coverage has no ceiling for the view (never probed, absent, or nothing materialized), it falls back to the existing ResolveAsync(NpgsqlDataSource, ...) probe-and-read path unchanged, so a cold or unknown coverage is never less correct than before.
  • The MCP tool (DarlingMcpTrendTools) and the desktop viewer (ViewerDataService.QueryTrends) both now fetch their already-cached coverage first and pass it in, instead of letting the resolver probe on its own.
  • Routing decisions and the raw-only fallback SQL are unchanged.

Test plan

Run on a local TimescaleDB rig (timescale/timescaledb:2.30.1-pg18, shared_preload_libraries=timescaledb,pg_stat_statements):

  • New live pin, QueryStoreTrendRoutingCachedLiveTests.ResolveAsync_WithACachedCeiling_IssuesNoBoundsProbe_AndMatchesTheFallbackRoute: seeds two hours of Query Store snapshots, materializes the rollup, resolves the route once through the uncached probe-and-read overload as the reference answer, then resolves it again through the coverage-taking overload and asserts the two routes agree (UseRollup, RawStartUtc, RollupFloorUtc) and that pg_stat_statements shows ZERO calls matching the min(bucket)/max(bucket) bounds statement after the cached call — the same pg_stat_statements counting discipline StoreSizeCacheLiveTests uses against the product's own call path, not a mock.
  • QueryStoreTrendRoutingTests, QueryStoreTrendRoutingLiveTests, QueryStoreTrendRoutingCachedLiveTests, RollupCoverageCeilingLiveTests, RollupCoverageRoutingTests, StorageCommandTimeoutTests, StartupCommandTimeoutTests: Total: 98, Failed: 0.
  • The test-census family (CommentFilterAdoptionTests, DocCommentHygieneTests, McpPayloadContractCensusTests, every *Census* class): Total: 355, Failed: 0.
  • McpToolsListBudgetTests: Total: 5, Failed: 0.
  • Build: Darling.Tests, the Darling Viewer, and the Darling Service each build with -p:EnableWindowsTargeting=true at 0 warnings, 0 errors.
  • Lite.Tests: no Lite/DuckDB code changes here, so no Lite class is affected.

CHANGELOG

SECTION: Changed
ENTRY: - The Darling Viewer's Query Store trend chart and get_query_store_duration_trend no longer re-scan the hourly rollup's oldest and newest hour on every load ([#4618]) - They take the rollup's range from the coverage the service already caches, instead of a fresh min/max read over the rollup on every call.
REF: [#4618]: #4618

erikdarlingdata and others added 7 commits September 28, 2026 12:53
…fresh min/max(bucket) read

The trend router (get_query_store_duration_trend and the desktop Query
Store chart) used to run min(bucket)/max(bucket) over the corrected
hourly rollup on every call to decide the raw/rollup boundary. Both
callers already hold a RollupCoverage from the coverage probe they
share with every other reader of the store, so the resolver now reads
its floor and measured ceiling from that cache first, and falls back
to the live probe-and-read only when the coverage carries no ceiling
for the view (cold or unknown coverage).

Routing decisions and the raw-only fallback are unchanged; the ceiling
IS the same exclusive boundary the pure decision computes from
max(bucket) + bucket width, so no adjustment is needed when reading it
from the cache.

Refs #4611. Builds on the rollup-ceiling change from #4605.
…roof

Pure pins for RollupCoverage.CeilingOf: an unknown view answers null, the
six-argument MergeRollupFloors overload replaces a re-measured view's
cached ceiling while a view reused from cache keeps its own, and the
five-argument overload keeps leaving every new entry's ceiling unset.

A live pin against a real TimescaleDB seeds six hours of raw query_store_stats,
refreshes the hourly rollup to now minus two hours, and checks CeilingOf
against the engine's own materialization watermark. A further refresh and a
forced re-measure show the ceiling advancing; an immediate follow-up call
inside the cache's reuse window shows it holding steady.

Reworded a comment that used 'ruling' to 'decision' for plain wording.
QueryStoreTrendRoutingCachedLiveTests proves the coverage-taking
overload of QueryStoreTrendRouting.ResolveAsync issues no fresh
min(bucket)/max(bucket) probe against query_store_stats_corrected_hourly
once a caller already holds a RollupCoverage with a measured ceiling
for that view, and that it reaches the same routing decision the
uncached probe-and-read overload does for the same store. Counted
through pg_stat_statements against the actual call, the same
discipline StoreSizeCacheLiveTests uses.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant