Skip to content

Compose: HardRowCap truncation on time series silently keeps the OLDEST slice and says nothing #1687

Description

@erikdarlingdata

Found while building the #1606 reporting-layer demo view, live on DARLING01. Pre-existing v2 behavior, not introduced by #1683.

Problem

A grouped time-series panel whose (buckets × groups) product exceeds ComposeLimits.HardRowCap (10,000) gets ORDER BY bucket + LIMIT 10000 — which keeps the earliest rows and drops the rest. The chart then renders a fraction of the requested window as if it were everything: a 24-hour wait_stats panel grouped by wait_type at minute grain (~100 types × 1440 buckets ≈ 144k rows) displayed exactly 7/25 11:57 AM → 1:24 PM — the first ~87 minutes — with no cap indicator anywhere. The x-axis even omits dates (the truncated domain doesn't cross midnight), so nothing looks wrong.

Same silent-truncation class as the retention-routing gaps (#1661/#1664/#1665): the numbers aren't wrong, the window is, and a monitoring product must say so.

Repro

{"source":"wait_stats","measure":"wait_time_ms","aggregate":"sum","timeBucket":"minute","groupBy":["wait_type"],"viz":"stacked"} over 24h on a fleet store with a normal wait-type population.

Fix directions (pick at review)

  1. Keep the NEWEST slice: the cap query becomes ORDER BY bucket DESC LIMIT n in a subquery re-ordered ascending — recent data is what a monitoring chart is for; still truncated, but the honest end.
  2. Say so: when rows == HardRowCap exactly, attach the existing notice mechanism ("row cap reached — showing the most recent N of the window; coarsen the bucket or narrow the group-by"). The Darling compose: CAGG routing has no rollup-availability gate — old-window custom-view panels throw 42P01 on plain-PostgreSQL stores #1665 notice plumbing (payload notice + noticeStrip) makes this a few lines.
  3. Optionally: the write-time bucket ceiling only bounds buckets, not buckets×groups — a validation hint when a grouped minute-grain panel is likely to cap would prevent authoring the trap at all.

(1)+(2) together are the honest minimum.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions