WPF Wait Stats and Perfmon trend charts bucket server-side (#4234)

[erikdarlingdata]

Part of #4234. (File I/O, memory clerks, the Overview lanes, Performance Trends and Lite are later lanes.)

Why

Lane T1 built the Wait Stats and Perfmon trend bucketing and the two name-list caches (build succeeded,
0 warnings) but never stood up a rig, wrote no tests, and ran no suite. The coordinator reviewed the diff
against the #4234 ruling and found two defects before any of that verification had happened. This PR
finishes the lane: fixes both defects, writes the ruling's four required tests plus an MCP-twin equality
pin, measures on a real rig, and runs the full suite.

What was wrong

1. Ruling item 3 ("when the budget covers every collection in the window, the chart gets the raw points
   unchanged") was not met. At a width that gave each collection its own bucket, WaitTrendsSql and
   PerfmonTrendsSql still stamped every point at its date_bin bucket start, never the raw
   collection_time — T1's own PR description called this out as a deliberate, accepted difference
   ("only the timestamp changes"). Proven live: seeding three collections five minutes apart with
   non-zero seconds (:37) and reading them back at the finest width returned 09:00:00, 09:05:00, 09:10:00 instead of the seeded 09:00:37, 09:05:37, 09:10:37.
2. ViewerNameListCache keyed a hit on (server, window length) alone. A 7-day custom range from last
   month could reuse this week's cached 7-day wait-type or counter list, because both windows share one
   length-keyed slot and nothing checked which window was actually fetched.

What changes

• WaitTrendsSql / PerfmonTrendsSql now also project first_collection_time (MIN(collection_time),
  mirroring DurationTrendRouting.BuildBucketedRawTrendSql's own column of that name) and
  collection_count (COUNT(*)) per bucket. GetWaitStatsTrendsByTypesAsync and
  GetPerfmonTrendsByCountersAsync buffer every row and, only when EVERY bucket the whole call returned
  holds exactly one physical collection, stamp each point at its bucket's first_collection_time instead
  of bucket_start — reproducing the pre-WPF trend charts never adopted the #3897/#3960 bucket contract: at the "Last 7 days" window one chart reads 85–170 k raw rows, and the pickers re-run a week-long DISTINCT every refresh #4234 read's timestamps exactly when nothing was merged, and
  falling back to bucket_start (what the MCP twins always serve) the instant any bucket in the call
  merged two or more collections. The MCP twins (WaitTrendBucketedSql, PerfmonTrendBucketedSql,
  DurationTrendRouting.BuildBucketedRawTrendSql) carry no such switch themselves — they stamp bucket
  starts at every width — so this is the least-invention reading of item 3 available: it reuses the
  twins' own first_collection_time metadata column rather than adding a new mechanism, and the decision
  is made once per whole call (not per series, not per row), which keeps it simple to state and to test.
• ViewerNameListCache.Entry now also carries the fetched window's WindowEndUtc. A hit requires BOTH
  the existing wall-clock freshness check AND the cached entry's window end to sit within the 15-minute
  TTL of the requested end (Duration(), checked both directions — a custom range's end can move earlier
  than the cached one, not just later like a sliding preset's auto-refresh always does).
  GetDistinctWaitTypesAsync / GetDistinctPerfmonCountersAsync thread endUtc through TryGet/Set.
• Two PRE-EXISTING tests (WaitTrendsSql_KeepsLitesPerTypeLagPerSecondMath,
  PerfmonTrendsSql_SumsValueAndDelta_ButTakesTheIntervalAsMax) pinned the old per-collection SQL's exact
  text and were broken by T1's original bucketing commit, silently, because the suite was never run.
  Fixed to pin the new bucketed shape instead of reverting the SQL.
• New Darling/Darling.Tests/ViewerTrendBucketingReviewTests.cs: the ruling's four required tests plus
  one equality pin (see Test plan).

Measurement (rig: PostgreSQL 18 + TimescaleDB, port 55973, UTC)

Seeded 7 days at the real 1-minute cadence (CollectorScheduleDefaults) for 20 wait types (201,620 rows,
close to the issue's cited 170,020) and 12 counters (120,972 rows, close to the issue's cited 102,012) on
a dedicated probe database. EXPLAIN (ANALYZE, BUFFERS), old SQL via git show origin/dev:<file>
against the new:

Read	Rows seeded	Old (pre-#4234) rows returned	Old exec time	New (bucketed, width=10min) rows returned	New exec time
Wait trend, 20 types	201,620	201,620	1,117 ms	20,180	1,530 ms
Perfmon trend, 12 counters	120,972	120,972	1,133 ms	12,108	788 ms
Wait-type DISTINCT (name list)	201,620	20 rows either way — SQL unchanged, item 4 changes only how often it runs	172 ms	(same SQL)	(same SQL)
Perfmon-counter DISTINCT (name list)	120,972	12 rows either way — SQL unchanged	43 ms	(same SQL)	(same SQL)

Both trend reads cut the row count returned by about 10x at this window/width, matching the issue's
complaint that every raw collection shipped to a chart that cannot draw more points than it has pixels.
Backend execution time is NOT uniformly faster — the wait trend's new plan does more CPU work per row
(an extra GroupAggregate layer over the same 201,620-row scan) and measured slightly slower in-database
despite returning 10x fewer rows; the perfmon trend measured both faster and smaller. The real win either
way is the payload actually sent to and rendered by the WPF chart, not raw backend CPU time, and that
drops by an order of magnitude on both reads. The name-list EXPLAIN is unchanged before/after, as
expected — ruling item 4 only changes how often it runs (at most once per 15 minutes now, was every
1-minute auto-refresh), never the SQL itself.

Test plan

• dotnet build Darling/PerformanceMonitor.Darling.Viewer/PerformanceMonitor.Darling.Viewer.csproj -c Debug
  — Build succeeded, 0 Warning(s), 0 Error(s).
• dotnet build Darling/Darling.Tests/Darling.Tests.csproj -c Debug — Build succeeded, 0 Warning(s),
  0 Error(s), after merging origin/dev.
• Source pin: WaitTrendsSql/PerfmonTrendsSql carry a bucket width (date_bin(CAST($n AS integer)...), proven once by hand to fail against the pre-WPF trend charts never adopted the #3897/#3960 bucket contract: at the "Last 7 days" window one chart reads 85–170 k raw rows, and the pickers re-run a week-long DISTINCT every refresh #4234 SQL: git show origin/dev:.../ViewerDataService.Waits.cs (and the Perfmon twin) contain zero occurrences of
  date_bin.
• Live pin: a 7-day, 2-series read returns at most budget × series rows
  (ViewerTrendBucketingLiveTests.WaitTrend_SevenDayWindow_ReturnsAtMostBudgetTimesSeriesRows and its
  Perfmon twin).
• Live pin: point equality when the budget covers every collection — raw timestamps (off the minute
  grid) AND values unchanged
  (ViewerTrendBucketingLiveTests.WaitTrend_BudgetCoversEveryCollection_ReturnsRawTimestampsAndValuesUnchanged
  and its Perfmon twin). Proven to fail on the pre-fix behavior by temporarily forcing
  everyBucketSingleton = false and reverting — both failed exactly as expected (09:00:00 instead
  of the seeded 09:00:37), confirmed passing again after revert.
• Cache pin: a second refresh inside 15 minutes runs no DISTINCT (ViewerNameListCacheTests, unit
  tests directly against ViewerNameListCache — hit within TTL, miss at 15 minutes, PLUS the defect-2
  pin: miss when the cached window's end is far from the requested end despite a fresh fetch, and the
  two-sided early-end case).
• Extra equality pin: for one wait type and one counter, the WPF read's bucket starts and values match
  DarlingDataReader.GetWaitBucketsAsync / DarlingTrendReader.GetPerfmonBucketsAsync at the same
  explicit width (60 minutes, multi-collection buckets, so this exercises the shared aggregation math
  rather than the singleton-collapsing wrapper the point-equality pins above already cover).
• Targeted classes (*ViewerWaitStats*, *ViewerPerfmon*, *ViewerTrendBucket*,
  *ViewerNameListCache*): 40/40 passed against the rig.
• git merge origin/dev — clean, no conflicts.
• Full Darling.Tests.exe run against a freshly dropped/recreated darlingtest, on a UTC rig, after
  the merge: Total: 14063, Errors: 0, Failed: 4, Skipped: 49, Not Run: 1 (914.8s). All 4 failures
  are outside this lane's files (TrendPayloadBudgetLiveTests, CaptureDownChunkOrderTests,
  ServerListAndSummaryPlanShapeTests, DarlingCliCommandsHostCheckTests — plan capture, plan
  shape, CLI host checks; nothing that touches Wait Stats, Perfmon, or the name-list cache). A first
  run (before a fix below) additionally failed DocCommentHygieneTests and NOT
  DarlingCliCommandsHostCheckTests — the failure set changed between two back-to-back runs on
  unchanged code, which is the signature of environmental flakiness (likely rig contention: the first
  run overlapped with the measurement seeding/EXPLAIN queries below on the same PostgreSQL instance),
  not a deterministic regression. Cross-checked against dev's latest COMPLETED CI Build run
  (36156144337): it fails a different, also-unrelated test (PgTargetBlockingTests), so none of my 4
  are freshly confirmed by CI either way — flagged here for the coordinator's census rather than
  filed as issues from this lane.
• Fixed in passing: DocCommentHygieneTests.NoMemberCarriesTwoStackedSummaryBlocks caught two stacked
  <summary> blocks T1 left behind in both WaitTrendsSql and PerfmonTrendsSql (the pre-WPF trend charts never adopted the #3897/#3960 bucket contract: at the "Last 7 days" window one chart reads 85–170 k raw rows, and the pickers re-run a week-long DISTINCT every refresh #4234
  summary never merged with the one added for bucketing). Merged into one with <para> breaks.

Note on scope

I did not add a raw-vs-bucketed SQL switch, because none exists anywhere in this family today — the MCP
twins (get_wait_trend, get_perfmon_trend, the duration-trend raw tier since #3897) all stamp
date_bin bucket starts unconditionally, at every width, and carry no "don't bucket" branch. Item 3 is
met here by reusing the twins' own first_collection_time metadata column as the point's stamp exactly
when nothing was merged, which is the smallest change that satisfies the ruling's literal wording without
inventing a mechanism the rest of the codebase doesn't have.

CHANGELOG entry

SECTION: Fixed
ENTRY:

• The Darling viewer's Wait Stats and Perfmon trend charts read time buckets, not every collection ([WPF Wait Stats and Perfmon trend charts bucket server-side (#4234) #4304]) - Both charts read every collection in the window on each refresh: for 20 wait types over 7 days at a 1-minute cadence, that was about 170,000 rows. They now read buckets sized to the chart's point budget, which cut the rows returned about tenfold for a 7-day window in the measurement. A bucket's rate is its summed wait (or counter delta) over its summed collection interval. When the budget covers every collection, the chart still gets each raw point at its own timestamp. The wait-type and counter pickers ran a full-window DISTINCT on every 1-minute refresh; they now reuse the list for up to 15 minutes for the same server and window.
  REF:
  [WPF Wait Stats and Perfmon trend charts bucket server-side (#4234) #4304]: WPF Wait Stats and Perfmon trend charts bucket server-side (#4234) #4304

🤖 Generated with Claude Code

https://claude.ai/code/session_01FVjn4PBJN71NQXdFo6ZxNQ