Bucket Lite's memory clerk and File I/O trend reads (#4234)

[erikdarlingdata]

Part of #4234.

This branch was built on top of #4331, which has now merged to dev (fa05e9fd). The merge of origin/dev (fd1fdb20) took dev's copy of LocalDataService.WaitStats.cs and LocalDataService.Perfmon.cs, which this PR does not change. The diff now shows only this PR's files: Lite/Services/LocalDataService.Memory.cs, Lite/Services/LocalDataService.FileIo.cs, Lite/Controls/ServerTab.Refresh.cs, Lite/Mcp/McpIoTools.cs, and the new Lite.Tests/LiteMemoryFileIoTrendBucketingTests.cs.

Why

Same ruling as #4331/#4304: Lite's memory-clerk and File I/O charts ran one query per collection over the whole visible window, so a 7-day view could return thousands of points per file/clerk-type series. This bucket those reads to the chart's point budget, the same GREATEST(time_bucket(...), $2) shape #4331 gave the wait-stats and perfmon reads, and caches the memory-clerk picker's name list the same way #4331 cached the wait-type picker's.

What changes

• GetDistinctMemoryClerkTypesForPickerAsync (new): wraps the existing uncached GetDistinctMemoryClerkTypesAsync with LiteNameListCache, keyed on (server, window length), TTL 15 minutes — exactly GetDistinctWaitTypesForPickerAsync's shape. Only the two picker call sites in ServerTab.Refresh.cs (sub-tab refresh and full refresh) move to it; the shared read stays uncached for any future MCP caller.
• GetMemoryClerkTrendsByTypesAsync: now buckets via a new MemoryClerkTrendsSql(clerkTypeCount). A clerk's memory is a gauge, so a bucket is the plain AVG(memory_mb) of its collections — no rated/unrated split, no HAVING, since Measurement-layer campaign: the delta honesty contract (11 findings, one keystone) #3540 only marks delta rows.
• GetFileIoLatencyTrendAsync: buckets via a new FileIoLatencyTrendSql. The top-10-files ranking (top_files) stays an unbucketed whole-window scan, unchanged. A rated CTE nulls (not filters) an unrated row's read/write/stall contributions, so an unrated collection still counts toward collection_count; HAVING COUNT(rated_reads) > 0 drops a bucket with no rated row. A bucket's latency is SUM(stall) / SUM(operations), never a mean of the collections' own ratios. This read also feeds the Overview correlated timeline's I/O lane (CorrelatedTimelineLanesControl.xaml.cs, both the live fetch and the comparison-period fetch) — that control just groups the already-bucketed points by CollectionTime and averages across files client-side, so it needed no changes of its own.
• GetFileIoThroughputTrendAsync: buckets via a new FileIoThroughputTrendSql. The LAG-derived interval (falling back for pre-v60 rows with no stored sample_interval_seconds) is still computed once over the raw per-collection rows, before bucketing. A bucket's rate is SUM(bytes) / SUM(rated seconds), time-weighted, never a mean of per-collection rates.
• GetTempDbFileIoTrendAsync: buckets the same way as the latency read (own TempDbFileIoTrendSql) since it feeds UpdateTempDbFileIoChart over the same windows. No top-10 ranking pass (tempdb's file count is already small) and no queued-latency columns (this chart never drew them). Preserves a pre-existing quirk exactly: the reader maps the SQL's file_name column into FileIoTrendPoint.DatabaseName, because that's the field UpdateTempDbFileIoChart groups its series on — documented in the new doc comment rather than changed, to keep this PR to bucketing only.
• Every read gets the same "when every bucket in the call holds exactly one physical collection, stamp at first_collection_time; otherwise bucket_start" singleton rule Bucket Lite's Wait Stats and Perfmon trend and picker reads (#4234) #4331 established.
• In-lane fix: a one-line comment in McpIoTools.cs called the desktop chart's GetFileIoLatencyTrendAsync "untouched: a chart wants every collection" — true before this PR, false after. Updated it to say the chart read is bucketed too now, on its own budget, separate from the MCP tool's own GetFileIoTrendAsync.

I left the FileIoTrendPoint.DatabaseName-holds-file_name naming quirk in GetTempDbFileIoTrendAsync alone rather than renaming it — fixing that touches a DTO shared with the top-10 latency read (where DatabaseName is used correctly) and a chart grouping call, and is unrelated to bucketing. Flagging it here rather than filing a separate issue, since it's cosmetic (the data is correct, just confusingly named) and not worth a queue entry.

Test plan

• dotnet build Lite.Tests/Lite.Tests.csproj: 0 Warning(s), 0 Error(s), both before and after the origin/dev merge.
• New file Lite.Tests/LiteMemoryFileIoTrendBucketingTests.cs (18 tests): source checks that each of the four new SQL statements carries its bucket-width parameter and projects first_collection_time/collection_count; live-DuckDB tests for the 7-day row-budget cap (one per read), point equality when every bucket holds one collection seeded off the minute grid (:37 seconds), an unrated collection alone in a bucket leaving that point absent (latency and throughput), a merged bucket proving SUM/SUM rather than a mean of per-collection ratios/rates (latency and throughput), and the clerk picker's TTL cache (hit inside 15 minutes, miss past it, shared read never caches).
• Proven against the pre-WPF trend charts never adopted the #3897/#3960 bucket contract: at the "Last 7 days" window one chart reads 85–170 k raw rows, and the pickers re-run a week-long DISTINCT every refresh #4234 shape: temporarily reverted LocalDataService.Memory.cs, LocalDataService.FileIo.cs and ServerTab.Refresh.cs to their HEAD~2 (pre-bucketing) content and rebuilt with the new test file in place. It fails to compile — GetDistinctMemoryClerkTypesForPickerAsync, MemoryClerkTrendsSql, FileIoLatencyTrendSql, FileIoThroughputTrendSql and TempDbFileIoTrendSql don't exist on the old shape at all, which is a stronger guarantee than a runtime assertion failure would be. Restored the bucketed files from my own commits (git checkout HEAD -- <paths>) afterward and confirmed the build is clean again.
• Targeted run: *LiteMemoryFileIoTrendBucket*, *Overview*, *CorrelatedTimeline*, *Memory*, *FileIo*, *TrendBucket*, *DocCommentHygiene* — 133 total, 0 failed.
• Merged origin/dev (brought in the Overview CPU/wait/memory lane bucketing from Bucket Lite's Overview CPU, wait and memory lane reads (#4234) #4337 and other unrelated work); rebuilt clean, no conflict markers left in any touched file.
• Full suite once, after the merge: Lite.Tests.exe — 5455 total, 0 errors, 0 failed, 0 skipped (386.9s).
• Not run: Installer.Tests (excluded per lane orders), and no live SQL Server/Postgres rig (Lite only, embedded DuckDB, per the brief).

CHANGELOG entry

SECTION: Changed
ENTRY:

• Lite's memory clerk and File I/O trend charts bucket long windows in DuckDB ([Bucket Lite's memory clerk and File I/O trend reads (#4234) #4340]) - The memory clerk, File I/O latency, File I/O throughput and TempDB File I/O trend charts, and the Overview I/O lane, read one point per collection. They now gather collections into buckets sized to the window, at most 1,500 points per series. A bucket's latency is its summed stall over its summed operations, and its throughput is its summed bytes over its summed seconds. A window short enough that every bucket holds one collection plots each point at its own collection time, as before. The memory clerk picker's name list is cached for 15 minutes.
  REF:
  [Bucket Lite's memory clerk and File I/O trend reads (#4234) #4340]: Bucket Lite's memory clerk and File I/O trend reads (#4234) #4340

🤖 Generated with Claude Code

https://claude.ai/code/session_01FVjn4PBJN71NQXdFo6ZxNQ

[erikdarlingdata]

Fixed the four DocCommentHygieneTests.NoMemberCarriesTwoStackedSummaryBlocks failures. Each stranded first block was the method's own summary, displaced above the SQL member this PR pulled out of it. Moved each verbatim to directly above its method, after the SQL member's closing line and the blank line, wording unchanged:

• Lite/Services/LocalDataService.FileIo.cs 73-86 -> above GetFileIoLatencyTrendAsync
• Lite/Services/LocalDataService.FileIo.cs 219-231 -> above GetFileIoThroughputTrendAsync
• Lite/Services/LocalDataService.FileIo.cs 344-352 -> above GetTempDbFileIoTrendAsync
• Lite/Services/LocalDataService.Memory.cs 255-267 -> above GetMemoryClerkTrendsByTypesAsync

All four moved line ranges matched what the CI failure and the brief pointed at exactly. git diff --stat: 2 files changed, 49 insertions(+), 49 deletions(-) — only the four blocks moved, nothing else touched.

One commit, 14a25915, pushed to fix/4234-lite-clerks-fileio-buckets (new head; was fd1fdb20).

Builds: Lite/PerformanceMonitorLite.csproj and Darling/Darling.Tests/Darling.Tests.csproj, both 0 Warning(s), 0 Error(s). Also built Lite.Tests/Lite.Tests.csproj (0 Warning(s), 0 Error(s)) to run its tests.

Tests:

• Darling.Tests.exe -class "*DocCommentHygiene*": Total 77, Failed 0, Errors 0.
• Lite.Tests.exe: the brief's class name LiteMemoryFileIoTrendBucketingTests isn't the actual class in that file — it holds LiteMemoryFileIoTrendBucketWidthSqlTests and LiteMemoryFileIoTrendBucketingLiveTests. Ran both with -class "*LiteMemoryFileIoTrendBucket*": Total 18, Failed 0, Errors 0.

No live test run (no rig requested). No full suite run. No Installer.Tests run.