Skip to content

Live visualizations for streaming queries: sliding-window aggregates and animated updates #61

Description

@BorisTyshkevich

Context

#59 covers the safe baseline for ClickHouse SELECT … STREAM: bounded memory,
throttled rendering, live lifecycle, Stop behavior, and table tailing.

This issue covers the next layer: making streaming results visually useful.

The goal is not just an appending table. The goal is live values that visibly move:
bars grow and reorder, pie slices resize, stat cells tick, and changed values are easy
to notice.

Problem

The ClickHouse stream is append-only raw rows. For live charts, the browser cannot
assume the server sends updated aggregate rows.

The browser needs a client-side rollup over a bounded window, then it needs to update
existing visualizations in place instead of rebuilding them for every chunk.

Proposed work

  • Add a pure client-side aggregation window in src/core/.
    • count-based window, for example last N rows;
    • optional time-based window, for example last 60 seconds when a timestamp column exists;
    • group-by accumulators for count/sum/avg/min/max where feasible;
    • deterministic eviction so memory stays bounded.
  • Reuse existing chart selection/pivot logic where possible, especially src/core/chart-data.js.
  • Update Chart.js instances in place with chart.update() instead of destroying/recreating the canvas per chunk.
  • Batch chart updates on the same render cadence as the streaming table.
  • Add live stat/table-cell affordances.
    • changed numeric cells briefly highlight or count up;
    • support prefers-reduced-motion;
    • show the active window, for example last 10k rows or last 60s.
  • Keep the raw tail table and aggregate chart views as two views over the same feed state.

Suggested architecture

Area Proposed location Notes
Window policy src/core/stream.js or src/core/stream-window.js Count/time window, eviction, received/dropped counters
Aggregation src/core/stream-aggregate.js Pure fold/evict logic
Chart model src/core/chart-data.js Reuse/extend existing chart preparation
Render scheduling src/core/stream.js + UI rAF glue Pure throttle decision; UI owns timers
Animation/UI src/ui/results.js, src/styles.css In-place chart updates, cell transitions, reduced-motion support

Acceptance criteria

  • A streaming query can feed a chart without unbounded memory growth.
  • Chart updates are batched and smooth under sustained row arrival.
  • Existing chart instances are updated in place where possible.
  • The UI makes the aggregation window explicit.
  • Aggregation, eviction, null handling, and update batching decisions have unit coverage.
  • Motion can be reduced through standard reduced-motion preferences.

Open questions

  1. Should the default chart window be last-N-rows, time-based, or user-selectable per query?
  2. Should dimension/measure selection reuse the current Chart view controls, or should live mode expose a simplified picker?
  3. Which aggregations are first scope: count only, count/sum/avg, or broader numeric aggregations?
  4. How should high-cardinality dimensions work: top-K with Other, hard cap, or explicit user choice?
  5. Should a STREAM LIMIT query use the live chart path while running and then settle into a normal finite chart when complete?

Non-goals

  • Enabling ClickHouse streaming settings on behalf of the user.
  • Server-side incremental aggregation.
  • A new charting dependency unless Chart.js proves insufficient.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions