Compose: keep the NEWEST buckets when the row cap truncates, and say so (#1687)

[erikdarlingdata]

Closes #1687. Both halves the issue called the honest minimum: keep the newest slice, and say when the slice happened.

The bug

A grouped time-series panel whose (buckets × groups) product exceeds the 10,000-row hard cap got ORDER BY bucket + LIMIT 10000 — which keeps the earliest rows. A 24-hour wait_stats panel grouped by wait_type at minute grain rendered its first ~87 minutes as though that were the whole window, and because the truncated domain never crossed midnight the x-axis showed no dates, so nothing on screen looked wrong.

Same silent-truncation class as the retention gaps (#1661/#1664/#1665): the numbers are right and the window is wrong.

1. Keep the newest buckets

The cap now applies ORDER BY bucket DESC inside a subquery, with the survivors re-sorted ascending for the renderer. Actual emitted SQL:

SELECT * FROM (
SELECT date_trunc('minute', f.collection_time) AS bucket, f.wait_type AS wait_type, CAST(SUM(f.delta_wait_time_ms) AS double precision) AS value
FROM collect.wait_stats AS f
WHERE f.collection_time >= $1
  AND f.collection_time <= $2
GROUP BY date_trunc('minute', f.collection_time), f.wait_type
ORDER BY bucket DESC
LIMIT 10000
) AS capped
ORDER BY bucket

Two things I was careful about, both pinned:

• The wrapper opens after any CTE, so a module-join panel keeps WITH m AS (...) at the top level rather than swallowing it into the subquery. Verified on the real query_stats/object_name output.
• No parameter positions move. The time-series LIMIT is a literal — only the Ranked arm binds a LIMIT parameter — and the wrapper merely brackets text whose parameters were already appended in order. Confirmed on the untouched Ranked path, which still ends ORDER BY value DESC / LIMIT $3.

2. Say so

When a time-series panel comes back sitting exactly on the cap, a notice rides the existing #1665 plumbing (payload["notice"]), rendered by the noticeStrip already on every web surface and carried verbatim through MCP:

row cap reached — showing the most recent 10,000 buckets of the window, not the whole of it. Coarsen the time bucket or narrow the group-by to fit the window into fewer rows.

It names both escape hatches, because "this was truncated" without "here is what to do about it" is only half a fix.

Exactly-at-cap is the only signal available. The compiler cannot know the product ahead of time — group cardinality is a property of the data, not the spec. So a panel that legitimately produces exactly 10,000 rows is reported as capped. That false positive is accepted deliberately: an unnecessary notice is cheap, and the reverse mistake is the bug this exists to fix.

Notices are joined, not prioritized. Retention and the row cap now truncate a panel from the same end, and a long grouped panel on a retention-active store hits both. Showing one and swallowing the other would under-report precisely the panel in the most trouble, so CombineNotices emits all that apply — extracted as its own function specifically so that behaviour is testable rather than inline.

Ranked and Scalar are untouched. A Ranked LIMIT is the caller's own topN, so reaching it is the request being honored, not truncation; warning there would be a false alarm on the most ordinary result there is.

The issue's optional third item — deliberately not implemented

A write-time hint would have to guess. MaxBuckets is 5,000 and HardRowCap is 10,000, so buckets alone can never trip the cap — capping always requires multiple groups, and group cardinality is exactly what a pre-flight check cannot know. Any such warning would be a heuristic firing on every grouped minute-grain panel, most of which are fine. The runtime notice covers the real case with real data instead. Flagging the reasoning rather than silently skipping it.

Validation — run against a real Postgres, not just string-matched

The existing pins use Contains, which is why they never caught this, so I did not rely on them. I stood up a throwaway PostgreSQL 17.10 cluster and ran the project's own live compose test, which executes a time-series panel end-to-end through RunComposedPanelAsync:

RunComposedPanel_OldWindow_AgainstPlainPostgres_RunsCleanOnRaw   Passed
DarlingComposeTests (full)   192 passed, 0 failed, 0 SKIPPED   <- the live-gated ones actually ran

Zero skips is the part that matters: the wrapped SQL was parsed and executed by a real Postgres, not just matched as a string.

Darling.Tests   3265 passed, 0 failed
Lite.Tests      1495 passed, 0 failed
Solution build  0 warnings, 0 errors

Six new tests: the DESC-limited subquery shape (including a DoesNotContain on the ascending-inner form, which would re-introduce the bug while still looking wrapped), the CTE-ordering pin, Ranked/Scalar staying unwrapped, notice-at-exactly-cap, no-notice-below-cap, time-series-only, and the combine-all-notices behaviour.

I did not touch charts.js — the notice mechanism is already rendered there.

I also borrowed the shared DarlingDevPg rig briefly before switching to a throwaway; it was not running when I found it and it is not running now.

🤖 Generated with Claude Code