Query heatmap resolves preview text for cell winners only, not every row (#4233)

[erikdarlingdata]

Closes #4233.

Why

The query heatmap's base CTE resolved and truncated query_text for every row in the window, through the query_text_dim join in v_query_stats. Then it threw almost all of that work away, because each (time_bin, bucket_index) cell shows only its rn = 1 row.

The cost was measured on a seeded rig. It held 180,000 query_stats rows over 24 hours and 1,400 distinct query_text_dim texts of about 700 bytes each. The read had a server_id filter and no database filter.

• Before: EXPLAIN (ANALYZE, BUFFERS) took 2035.5 ms, with shared hit=7668, temp read=2186 written=2190. The plan ran a Nested Loop Left Join to query_text_dim once per fact row (179,875 probes). It also sorted rows of about 751 bytes, and that sort spilled to disk (an external merge of about 14 MB across workers).
• After: EXPLAIN (ANALYZE, BUFFERS) took 241.9 ms, with shared hit≈6924, temp read=1615 written=1618. The Index Scan on query_text_dim_pkey ran 1,012 times (loops=1012), once per winning cell instead of once per row. The sorted rows were 88 to 96 bytes wide instead of about 751.

That is about 8 times faster on this rig. The issue's own production numbers, 613 ms to 266 ms, came from warmer buffers and real Query Store data. The change is the same in both cases. The join and the truncation no longer run once per row, and only a narrow key goes through the sorts.

What changes

Both copies of BuildQueryHeatmapSql change the same way: the desktop viewer's chart, and the MCP and web reader behind get_query_heatmap. The differences between the two copies that were already there stay.

• base now reads query_stats directly, never v_query_stats (query_text/query_plan_xml stored inline per row: 94% of a field store — normalize into hash-keyed dimension tables (~135x measured) #1767's view that resolves the payload). It carries query_hash, the inline query_text and query_text_digest through base, binned and ranked. Only rows from before query_text/query_plan_xml stored inline per row: 94% of a field store — normalize into hash-keyed dimension tables (~135x measured) #1767 have an inline query_text. Before, base carried a preview column that was already resolved and truncated.
• The outer SELECT ... FROM ranked WHERE rn = 1 resolves the preview only for the row each cell keeps. It uses LEFT(COALESCE(query_text, (SELECT d.query_text FROM query_text_dim d WHERE d.digest = ranked.query_text_digest)), <n>). That is the resolution v_query_stats does, moved to after the rn = 1 filter.
• <n> is the literal 120 in the viewer's copy, and the bound parameter $7 in the MCP reader's copy. MCP read tools have no default response-size budget: at default arguments 12 tools return >50 KB and 3 return >100 KB for one server, more than an agent client's per-result cap #4198 made the reader's preview width a parameter, and this PR keeps that parameter in the same place.
• DarlingQueryHeatmapReader.HeatmapCoverageSql moves its two EXISTS probes from v_query_stats to query_stats too, so the probe reads the same table as the query. The answer cannot change. v_query_stats has no WHERE clause and no join that changes the row count. So a row exists in one exactly when it exists in the other.
• The doc comments on both BuildQueryHeatmapSql methods and on HeatmapCoverageSql now describe the new shape and cite Query heatmap resolves and sorts query text for every query-stats row in the window, then keeps it for ~1 in 125: 613 ms vs 266 ms with the text fetched for the winners only (WPF viewer and get_query_heatmap) #4233. The old ones said that the preview was resolved from v_query_stats, which is no longer true.
• Lite does not change. See "Lite" below.

The results do not change: the same cells, counts, top hash and preview text. When two rows tie for a cell, PostgreSQL returns either one, as before. The live tests below check this.

Lite

Two Lite files read the heatmap: Lite/Services/LocalDataService.QueryHeatmap.cs (for MCP) and Lite/Services/LocalDataService.QueryStats.cs near line 1441 (the desktop chart's own copy). Neither one carries a resolved preview through a window sort, as the Darling copies did.

• DuckDB has ARG_MAX. So both Lite copies pick the winning row with ARG_MAX(query_preview, delta_execution_count) AS top_query_text in a plain GROUP BY, not with a ROW_NUMBER() window and WHERE rn = 1. ARG_MAX has to see the value of every candidate row to pick the winner. So Lite has no way to resolve the text after a filter without adding back the window query that ARG_MAX avoids.
• Lite has no query_text_dim table. It stores query_text inline on every query_stats row, so it has no join to remove. The cost in this issue, a join per row to a table keyed by hash plus a truncation per row, does not exist in Lite.
• In DuckDB, v_query_stats does not resolve anything either. DuckDbInitializer builds it as a plain SELECT * FROM query_stats, or, when parquet archiving is on, as a UNION ALL BY NAME with the archived parquet files. A Lite read of query_stats alone, as in the Darling fix, leaves out the archived rows.

Tests

• QueryHeatmapSqlTwinParityTests.BothCopies_AreByteIdentical_ThroughTheRnFilter (new, 5 cases, one per metric) calls both BuildQueryHeatmapSql methods directly. It removes the two differences that predate Query heatmap resolves and sorts query text for every query-stats row in the window, then keeps it for ~1 in 125: 613 ms vs 266 ms with the text fetched for the winners only (WPF viewer and get_query_heatmap) #4233: the bin width (a literal against $5) and the preview width (a literal against $7). It checks that both of those replacements happened. It cuts both strings at the final WHERE rn = 1, because the ORDER BY and LIMIT after it differ on purpose (HeatmapSql_CapsFromTheRecentEnd pins that). Then it checks that the rest is identical. With one space removed from the viewer's copy, all 5 cases failed and showed the position of the difference. With the space put back, they passed.

• DarlingQueryHeatmapLiveTests.OldSqlFromDev_AndTheNewBuilder_AgreeOnCellsCountsHashAndPreview (new) runs the SQL from before this PR and the new BuildQueryHeatmapSql against the same seeded window, with the same parameters. The old SQL is a literal in the test, taken from dev before this PR. The window has three cases, each in its own 5-minute bin:

  1. A winner whose text is only in query_text_dim (a digest, and no inline query_text).
  2. A winner from before query_text/query_plan_xml stored inline per row: 94% of a field store — normalize into hash-keyed dimension tables (~135x measured) #1767 (inline query_text, and no digest).
  3. A tie: two rows in one cell with the same delta_execution_count.

  For every cell, the two queries must return the same bin, bucket and count. For cases 1 and 2, they must also return the same hash and preview, and those must match what the test seeded. For case 3, each query's winner must be one of the two tied rows, with the hash and preview of that same row. The two queries can pick different rows. The test cleans up through LiveStoreCleanup.RunAsync.

Test plan

• Darling.Tests builds with 0 warnings, before and after the merge of dev.

• DarlingQueryHeatmapSurfaceAndSqlTests (27), ViewerQueryHeatmapSqlTests (8), QueryHeatmapSqlTwinParityTests (5), DarlingQueryHeatmapLiveTests (2) and ViewerQueriesRestLivePostgresTests (4): 46 of 46 passed against the rig.

• EXPLAIN (ANALYZE, BUFFERS) before and after, on the seeded rig above: 2035.5 ms to 241.9 ms.

• The full Darling.Tests suite ran once, after the merge of dev (d7fa0e48), against a new darlingtest. It ran 14,033 tests: 2 failed, 49 skipped and 1 not run. Neither failure is in a file this PR changes:

  • PayloadDimensionLiveTests.PreFilter_SkipsLockingFreshRows_ButStillRefreshesStaleOnes_AndInsertsNewDigests expected almost no WAL for a batch of fresh rows, and saw 105,768 bytes. WAL from the rest of the suite can reach that measurement. The rig's log showed a slow checkpoint from earlier work.
  • TrendPayloadBudgetLiveTests.EveryDefaultAnswer_StaysNearTheBudget_AndTheLargestAnswerStaysUnderTheCap got "Exception while reading from stream" from get_file_io_trend.

  On a new darlingtest, both classes then ran alone and passed: 19 tests, 0 failed.

• Lite: no Lite file changes. CI runs the Lite suite.

• The tie check (88740dd3): a temporary tie-break in the new builder made it pick the other tied row. The old check then failed on the tie cell (Expected: "0xTIEA", Actual: "0xTIEB"), and the new check passed. With the tie-break removed, DarlingQueryHeatmapLiveTests, QueryHeatmapSqlTwinParityTests and DocCommentHygieneTests ran 84 tests, 0 failed. DarlingQueryHeatmapSurfaceAndSqlTests ran 27 tests, 0 failed.

• GitHub Actions CI on 88740dd3: build, Darling PostgreSQL tests and Lite tests all pass.

CHANGELOG entry

SECTION: Fixed
ENTRY:

• The query heatmap loads much faster on large windows ([Query heatmap resolves preview text for cell winners only, not every row (#4233) #4295]) - The heatmap read resolved the query text of every row in the window, then kept one row per cell. Now it resolves the preview text only for the row that each cell shows. The cells, counts, top queries and preview text do not change. On a seeded window of 180,000 rows over 24 hours, the read went from 2,035 ms to 242 ms.
  REF:
  [Query heatmap resolves preview text for cell winners only, not every row (#4233) #4295]: Query heatmap resolves preview text for cell winners only, not every row (#4233) #4295

🤖 Generated with Claude Code

https://claude.ai/code/session_01FVjn4PBJN71NQXdFo6ZxNQ

[erikdarlingdata]

Fix (test only, no product SQL touched)

Darling/Darling.Tests/DarlingQueryHeatmapTests.cs, DarlingQueryHeatmapLiveTests.OldSqlFromDev_AndTheNewBuilder_AgreeOnCellsCountsHashAndPreview (now around line 636-706). Commit 88740dd, pushed to fix/4233-heatmap-preview-winners-only (was d7fa0e4, now 88740dd).

Kept TimeBin, BucketIndex, Count equal for every cell. Added tieBin/tiedPreviewByHash and, for the tie cell only, replaced the equality check on Hash/Text with: each query's winner must be one of 0xTIEA/0xTIEB, and its Text must match the preview that belongs to whichever hash it picked. Every other cell (the dimension-resolved winner and the pre-#1767 inline winner) still asserts full equality. Updated the doc comment's item (c) to say the old and new SQL are not promised to pick the same tied row, instead of claiming they scan in the same order. Left the existing tieCell checks (Count == 2, hash is one of the two) alone. No product SQL changed anywhere in the diff.

Diagnosis confirmed on the rig

Both queries rank a tie with ROW_NUMBER() OVER (... ORDER BY delta_execution_count DESC) and no second key. A probe assertion (temporary, not committed) showed the old SQL and the new builder currently agree by coincidence of plan (old=0xTIEA new=0xTIEA) - the flake the brief described hasn't fired yet, but nothing guarantees it won't.

Proof (both runs)

1. Old assertion is fragile. Reverted the test file to d7fa0e4 (git checkout --), then added a temporary secondary sort key to the NEW builder only (DarlingQueryHeatmapReader.BuildQueryHeatmapSql, ORDER BY delta_execution_count DESC, query_hash DESC) so it picks the other tied row than the old SQL's natural pick. Built and ran the single test: it failed exactly on the tie cell, as expected:

   Darling.Tests.DarlingQueryHeatmapLiveTests.OldSqlFromDev_AndTheNewBuilder_AgreeOnCellsCountsHashAndPreview [FAIL]
     Assert.Equal() Failure: Strings differ
     Expected: "0xTIEA"
     Actual:   "0xTIEB"
   DarlingQueryHeatmapTests.cs(681,0)
   Total: 1, Errors: 0, Failed: 1
   

2. New assertion is not fragile. Reapplied my fix on top (test file only), same temporary product tiebreak still in place, rebuilt, reran the same test: passed (Total: 1, Errors: 0, Failed: 0).
3. Removed the temporary tiebreak (git checkout -- on DarlingQueryHeatmapReader.cs). git diff --name-only showed only Darling/Darling.Tests/DarlingQueryHeatmapTests.cs before committing.

Build and final test totals (rig: 127.0.0.1:55991, UTC, darlingtest)

dotnet build Darling/Darling.Tests/Darling.Tests.csproj -c Debug: Build succeeded, 0 Warning(s), 0 Error(s).

With the clean tree (my fix, no product changes), Darling.Tests.exe:

• DarlingQueryHeatmapLiveTests (holds the fixed test) + QueryHeatmapSqlTwinParityTests (name given in the brief; it's actually the viewer-vs-MCP-reader SQL-text twin, unrelated to this test but run per the brief) + DocCommentHygieneTests: Total: 84, Errors: 0, Failed: 0, Skipped: 0.
• DarlingQueryHeatmapSurfaceAndSqlTests, the other class in the same file (lane-orders: filter by every class in a touched file, not just the file's name): Total: 27, Errors: 0, Failed: 0.

No full suite run locally; CI runs it. Never ran Installer.Tests.

Note on the brief's class name

The brief said "the class that holds this test" is QueryHeatmapSqlTwinParityTests. That name exists in a different file (QueryHeatmapSqlTwinParityTests.cs) and tests something unrelated (the viewer's copy of the SQL builder text vs. the MCP reader's copy). The test I actually fixed, per the brief's own file:line (DarlingQueryHeatmapTests.cs, near line 636), lives in DarlingQueryHeatmapLiveTests. Ran both classes, plus the sibling DarlingQueryHeatmapSurfaceAndSqlTests in the same file, plus DocCommentHygieneTests - all pass.

Rig

PostgreSQL 18.6 + TimescaleDB, port 55991, C:\GitHub\worktrees\rig-4295tie, UTC. Stopped with pg_ctl -m fast stop after the final run.

Nothing deferred; nothing filed.