Skip to content

Add Query Store total-CPU/total-duration compose measures, closes #2732 - #2738

Merged
erikdarlingdata merged 2 commits into
devfrom
fix/2732-qs-total-cpu
Aug 31, 2026
Merged

erikdarlingdata merged 2 commits into
devfrom
fix/2732-qs-total-cpu

Conversation

@erikdarlingdata

Copy link
Copy Markdown
Owner

Closes #2732.

What

The compose catalog's query_store_stats roster could rank by executions, a per-interval peak, or a per-execution average — never by what a query group actually consumed, so "what burned the most CPU over this window" was not expressible against the box's headline dataset. This adds qs_total_cpu_us and qs_total_duration_us.

How

A new measure shape, MeasureRatioMode.WeightedSum — the Weighted ratio's numerator with no denominator, exactly the fix direction from the issue:

  • Raw route (ComposeCompiler.BuildValueExpr): SUM(f.avg_cpu_time_us * f.execution_count) — one new arm beside the Weighted arm, no NULLIF (there is no division). Rides the same Query Store aggregates double-count re-collected cumulative intervals - tier 2: interval identity + CAGG rebuild (tier 1 read-side dedup shipped in #1845) #1841 dedup wrapper as everything else on this source.
  • CAGG routes (ComposeCaggValueMapper): SUM(f.cpu_us_weighted_sum) / SUM(f.duration_us_weighted_sum) — the corrected hourly/daily rollups already materialize the product-sums, and the totals re-aggregate additively, so CanRemapQueryStore admits the mode and BuildQueryStoreExpr reads the pre-multiplied column alone via the existing QsWeightedSumColumn map.
  • Catalog: the two measures ride MeasureKind.Ratio (the aggregation is the definition, so ValidAggs is empty like a ratio's) reusing the qs_avg measures' WeightedValueColumn/WeightColumn pair. That choice means ComposeSpec and the wire catalog need no new special case — referenced as ratio, aggregate fixed, kind: "ratio" on the wire — and time-series, ranked (top query_hash by total CPU), and scalar modes all work for free. Default unit s rather than ms: a window total across a store runs to seconds-to-hours.

No migration, no collector change, no backfill.

Tests

  • Compile_WeightedSum_IsProductSum_WithNoDenominator (both keys) — raw product-sum, no NULLIF, µs→s conversion.
  • Compile_OldWindow_QueryStore_WeightedSum_RoutesToCagg_ReadsTheWeightedSumColumnAlone — corrected-hourly remap, and never the raw column.
  • WeightedSumQsTotals_ValidateAsRatios_AndRejectMeasureReference + WeightedSumQsTotals_SupportRankedMode_TopQueryHashByTotalCpu — the spec seam and the issue's headline panel.
  • ComposedQueryStoreTotalCpuPanel_RunsOnPostgres_AndSumsTheProductOverDedupedIntervals (live, DARLING_TEST_PG-gated) — pins 500 × 41 = 20500 µs over the deduped fixture; an un-deduped product-sum would read 26500.
  • ComposeCaggValueMapperTests: CanRemap rows for both keys plus exact-expression facts for the weighted-sum columns.
  • Catalog pins updated: EveryRatioMeasure_ReferencesRealSameSourceScalars now filters to Sum/Avg modes; WeightedRatio_Columns_AreRealPayloadColumns covers both weighted modes (so the new measures' columns stay pinned to the collector's payload).

Verified locally on macOS with a throwaway net10.0 harness against the real Service assembly (the Windows test suites can't run here): 30/30 checks pass — catalog shape, spec validation, raw/hourly/daily compiled SQL, mapper expressions byte-exact, ranked mode, and the wire catalog (kind: "ratio", defaultAggregate: null, QS roster now 7 measures). CI is the arbiter for the xUnit suites.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JgLaaxFenZFZs1SntKQDEg

The compose catalog's query_store_stats roster could rank by executions,
a per-interval peak, or a per-execution average - never by what a query
group actually consumed. qs_total_cpu_us / qs_total_duration_us fill the
hole as MeasureRatioMode.WeightedSum: the Weighted ratio's numerator with
no denominator - SUM(avg * execution_count) on the raw route (riding the
same #1841 dedup wrapper), and the corrected rollups' pre-multiplied
cpu_us_weighted_sum / duration_us_weighted_sum columns directly on the
CAGG routes. Riding the Ratio kind means the spec/wire seam needs no new
special case: referenced as 'ratio', aggregation fixed, and time-series,
ranked, and scalar modes all work unchanged. No migration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgLaaxFenZFZs1SntKQDEg
@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Reviewed. This is a clean, additive change to the Darling compose catalog (no Lite counterpart exists for this feature, so no Lite/Darling parity concern applies here — Lite has no MeasureCatalog/ComposeCompiler).

Went through each MeasureRatioMode switch site touched by the new WeightedSum mode (ComposeCompiler.BuildValueExpr, ComposeCaggValueMapper.CanRemapQueryStore/BuildQueryStoreExpr) and confirmed all of them were updated in tandem — no site still only checks == Weighted while silently falling through for WeightedSum in a way that would break. QsWeightedSumColumn is correctly reused for both qs_total_* measures since they share the same value columns as the existing qs_avg_* measures. Unit math checks out: us -> s is a 1,000,000 factor in FamilyDuration, matching the / 1000000.0 the tests pin. The raw-route product-sum rides the same #1841 dedup wrapper per the new live Postgres test, and the CAGG route reads the pre-multiplied column with no NULLIF (correct, since there's no division). No new dynamic SQL construction — table/column names are catalog constants, values stay parameterized.

No correctness, parity, or security issues found. Nothing to flag.

… and #2710 entries)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgLaaxFenZFZs1SntKQDEg
@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown

Reviewed. This is a clean, well-scoped change — no T-SQL, no collector, no migration, no Lite files touched, so Lite/Darling parity is not a concern here (Lite has no compose/web-panel feature to keep in sync).

Checked specifically:

  • MeasureRatioMode.WeightedSum is handled everywhere Weighted is handled (ComposeCompiler.BuildValueExpr, ComposeCaggValueMapper.CanRemapQueryStore/BuildQueryStoreExpr) — no missed switch arm elsewhere in the service or frontend (compose.js is generic over ratio/unit, no mode-specific branching needed there).
  • QsWeightedSumColumn mapping (avg_duration_us → duration_us_weighted_sum, avg_cpu_time_us → cpu_us_weighted_sum) is correct and reused, not duplicated.
  • The new raw-route expression sits in the same code path as the existing Weighted arm, so it rides the same Query Store aggregates double-count re-collected cumulative intervals - tier 2: interval identity + CAGG rebuild (tier 1 read-side dedup shipped in #1845) #1841 per-interval dedup wrapper — consistent with the live-gated test's expected 20500 (deduped) vs 26500 (undeduped) values.
  • cpu_us_weighted_sum/duration_us_weighted_sum CAGG columns are pre-existing (from prior corrected-rollup work in TimescaleSupport.cs), so the "no migration" claim holds.
  • ComposeSpec.cs validation branches only on MeasureKind.Ratio, not on RatioMode, so no special case was needed for the new mode, matching the PR description.
  • Unit s (1,000,000 native-to-default factor) is a pre-existing ComposeUnit entry, not new machinery.

No correctness, security, or parity issues found.

@erikdarlingdata
erikdarlingdata merged commit c7d27f9 into dev Aug 31, 2026
6 checks passed
erikdarlingdata added a commit that referenced this pull request Aug 31, 2026
…-keys

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JgLaaxFenZFZs1SntKQDEg
@erikdarlingdata
erikdarlingdata deleted the fix/2732-qs-total-cpu branch September 12, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant