Skip to content

Route get_top_procedures_by_cpu and the Top Procedures grid to the hourly rollup (#4231) - #4413

Merged
erikdarlingdata merged 2 commits into
devfrom
fix/4231-top-procedures-hourly-routing
Sep 26, 2026
Merged

erikdarlingdata merged 2 commits into
devfrom
fix/4231-top-procedures-hourly-routing

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

Refs #4231.

Why

get_top_procedures_by_cpu and the Viewer's Top Procedures grid read procedure_stats directly, which a store with the rollups armed drops after a few days. Once the window ages past raw's floor the read should fall back to the hourly rollup rather than silently returning nothing, the same fix already shipped for get_top_queries_by_cpu (#4396).

What changes

  • A new DarlingDataReader.TopProceduresHourlySql, grouped by (database_name, schema_name, object_name) and ranked by SUM(worker_time_sum) DESC, with its FROM clause built only through RollupCoverage.StitchedRelationSql — never a literal procedure_stats_hourly name.
  • GetTopProceduresByCpuRoutedAsync picks Raw or Hourly via RetentionTierRouter.Resolve over RollupCoverage.For(ProcedureStatsHourlyView, ProcedureStatsDailyView); a Daily verdict is clamped to Hourly (out of scope for this change). GetTopProceduresByCpuAsync is now a thin wrapper over the routed read, so every existing caller keeps working.
  • The get_top_procedures_by_cpu MCP payload gains tier_used ("raw"/"hourly") and precision_note. The rollup carries no object_type, sql_handle, or plan_handle, so an hourly-routed row's object_type is reported as null (never the empty-string default) with a note explaining why.
  • The raw-window-floor probe and disclosure (window_truncated/effective_start) now run, and are meaningful, only for the raw tier — an hourly-routed read never touched procedure_stats, so probing its floor would answer a fact about a table the read never consulted.
  • The Viewer gets ViewerDataService.GetTopProceduresByCpuTierAsync (routes the same way, over the same coverage pair) and the Top Procedures grid header now appends the hourly-tier disclosure suffix when routed, exactly like the Queries grid already does.

Deviations

  • None. procedure_stats has no host_object_name and no query-text lookup, so this change is smaller than stage 3a: no follow-up per-row text query, no rollUpByHostObject/min_dop parameters to disable at the hourly tier.
  • The stage 3a live test's cache-freshness finding (ComposeStoreAvailability caches rollup coverage per NpgsqlDataSource for 5 minutes) applies identically here — the new live test uses a fresh data source for its post-refresh, post-purge call, same as TopQueriesHourlyRoutingLiveTests.

Test plan

  • TopProceduresHourlyRoutingTests (new, source-level): SQL-shape pin (GROUP BY database_name, schema_name, object_name, no object_type, ORDER BY SUM(worker_time_sum) DESC, $FROM$ placeholder present) and a source pin that GetTopProceduresByCpuHourlyAsync builds its FROM clause only through RollupCoverage.StitchedRelationSql, never naming procedure_stats_hourly/procedure_stats_interval_hourly literally. RED on dev confirmed: building this file against commit 30760481a (dev tip, stage 3a merged) fails to compile — DarlingDataReader.TopProceduresHourlySql/TopProceduresHourlyFromPlaceholder do not exist. GREEN on this branch: 2/2 passed in-process (Darling.Tests.dll on macOS).
  • TopProceduresHourlyRoutingLiveTests (new, live, own scratch database per Six live Darling.Tests classes run outside the live-postgres collection, producing moving cross-class flakes on a shared store #1776): seeds three (schema, object) groups into raw procedure_stats, confirms a routed call reports RetentionTier.Raw; refreshes procedure_stats_interval_hourly via the product's own refresh_continuous_aggregate, purges raw's rows for the window, and confirms a routed call on a fresh NpgsqlDataSource reports RetentionTier.Hourly with totals matching raw's for every group. A second test drives get_top_procedures_by_cpu end to end and asserts tier_used/precision_note on both tiers, and that object_type is null on every hourly-routed row. RED on dev confirmed: same commit, same compile failure (GetTopProceduresByCpuRoutedAsync does not exist). GREEN on this branch, against a throwaway TimescaleDB 2.30.1 container: 2/2 passed.
  • Regression: TopQueriesHourlyRoutingTests (2/2) and TopQueriesHourlyRoutingLiveTests (2/2) both still pass unchanged.
  • Both Darling.Tests.csproj and Lite.Tests.csproj build 0 errors / 0 (unrelated pre-existing) warnings with -p:EnableWindowsTargeting=true.
  • Darling.Tests/Lite.Tests target net10.0-windows and cannot run under CI on this machine's OS; they build here and CI decides them on Windows.

Measurement

Seed: 10,081 rows at 1-minute grain over 500 (schema, object) groups across a 7-day window (2026-01-01–2026-01-08), matching stage 3a's seed scale. EXPLAIN (ANALYZE, BUFFERS) for the ranking query, 1 cold + 3 warm runs, against a standalone table pair built directly from the product's own SQL shapes (not through migrations, to keep the measurement fast):

  • Raw (procedure_stats-shaped table): Seq Scan + HashAggregate, 206 buffer hits, Execution Time 2.79 ms (cold).
  • Hourly (procedure_stats_hourly-shaped table, pre-aggregated at the same grain for this seed): Seq Scan + HashAggregate, 156 buffer hits, Execution Time 2.33–3.06 ms (3 warm runs).
    At this seed scale the two are comparable because the hourly rollup here holds the same row count as raw (no compression from bucketing distinct minutes into hours happened in this synthetic seed); the real-world win is that raw ages out entirely after its retention window while the hourly rollup keeps answering, which is what the live routing test proves.

CHANGELOG entry

SECTION: Changed
ENTRY:

…urly rollup (#4231)

Refs #4231. Copies stage 3a's routing pattern (#4396) for procedure_stats: TopProceduresHourlySql over procedure_stats_hourly/procedure_stats_interval_hourly via RollupCoverage.StitchedRelationSql, GetTopProceduresByCpuRoutedAsync exposing the tier, the MCP tool's tier_used/precision_note, and the Viewer's tiered read plus grid header suffix.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant