Dashboard tiles: migrate onto the shared streaming read seam (app.runReadInto)

[BorisTyshkevich]

Deferred from #185

#185 added a shared streaming execution seam app.runReadInto(result, { sql, format, rowLimit, params, signal, queryId, onChunk }) (wraps ch.runQuery + applyStreamLine, writes no tab/global state) and moved the workbench run() and the new interactive detached Data view onto it — full streaming, server-side row cap, and real AbortController cancellation.

The dashboard tiles (src/ui/dashboard.js runSlotTile → app.runTile → queryDashboardTile) were intentionally left on their existing path in that PR. This issue migrates them onto the shared seam, gaining streaming transport, bounded client memory, progress, and real per-tile cancellation.

Why it is not a clean swap

• queryDashboardTile (src/net/ch-client.js) sets readonly: 2 — a deliberate server-side write-guard for dashboards — plus max_result_bytes (DASH_TILE_BYTE_CAP) and a DASH_TILE_ROW_CAP + 1 sentinel. runQuery (streaming) sets none of these.
• The tile result shape is parseJsonResult → {columns, rows, meta:{rows, ms, bytes, truncated}} and tileFooter(meta); the streamed newResult uses {columns, rows, progress, capped, cancelled, error}.
• Dashboards fan out up to TILE_CONCURRENCY (6) tiles via runPool, and runAffected() (filter re-runs) currently uses a direct Promise.all.
• applyTileResult() is a final-result classifier (0/1-row unconfigured tiles are hidden as future-KPI skips; explicit panels never vanish) — it must NOT be called per streamed chunk.

---

Design requirements (from review)

1. Row cap — keep the truncation guarantee

Request one sentinel row past the cap and let the client trimmer flag truncation:

const result = newResult('Table', DASH_TILE_ROW_CAP);      // client limit = CAP
await app.runReadInto(result, {
  sql: execSql,
  format: 'Table',
  rowLimit: DASH_TILE_ROW_CAP + 1,                          // server max_result_rows = CAP + 1
  params: { readonly: 2, max_result_bytes: DASH_TILE_BYTE_CAP, ...mergedSourceArgs(src) },
  signal, onChunk,
});

applyStreamLine trims against result.rowLimit = CAP and sets capped when the sentinel row arrives — distinguishing "exactly CAP" from ">CAP" without changing runReadInto/runQuery.

Note (workbench precedent): the workbench + detached view already detect capped via result_overflow_mode:'break' overshooting the block boundary (no +1 sentinel — same value for server and client), and it works. The CAP+1/CAP split above is a stricter belt-and-suspenders; either is acceptable. Don't treat the +1 as mandatory — just don't send max_result_rows = CAP and then expect exact-cap detection without relying on overshoot.

2. Token freshness — one preflight per wave

runReadInto leaves token freshness to the caller. Full Refresh already calls ensureFreshToken() once before the fan-out; runAffected() must do the same preflight before launching affected tiles (today app.runTile masks this by calling getToken() itself — removing that wrapper without the preflight would make affected tiles independently race a rotating-token refresh through authedFetch). A failed preflight follows the same sign-out behavior as full Refresh and issues no tile requests.

3. Reserve the generation when the wave is created, not when a pool worker starts

runPool starts only 6 workers; a queued tile does not increment its generation until a worker frees up. This allows a stale-wave race:

Full Refresh A queues tile 8 → user changes a filter → affected wave B runs tile 8 (newest) → Refresh A's worker finally reaches tile 8 and supersedes B with older values.

Reserve generation + abort at wave creation:

function supersedeSlot(slot) {
  const generation = ++slot.gen;
  if (slot.abortController) slot.abortController.abort();
  slot.abortController = null;
  return generation;
}
// full refresh:
const plan = targets.map(({ slot, ...rest }) => ({ ...rest, slot, generation: supersedeSlot(slot) }));
// queued worker, before issuing a request:
if (slot.gen !== generation) return;

4. Define streaming UI behavior — do NOT call applyTileResult per chunk

Calling the classifier on every onChunk would transiently hide chart tiles after the first row, destroy/recreate Chart.js per chunk, show incomplete footers, and briefly show partial data on a late error. Contract:

• onChunk updates only the tile's loading/progress placeholder (e.g. Loading… 1,420 rows);
• final panel classification + rendering happen once, after a successful, current-generation completion.

This keeps streaming transport, bounded memory, progress, and cancellation without continuously rebuilding charts. (Progressive chart/table rendering can be a separate, performance-tested feature.)

5. Explicit FORMAT — pick a policy (recommended: reject)

The streaming parser only understands JSONStringsEachRowWithProgress; an explicit FORMAT JSON/CSV/Pretty clause overrides the default and yields a wrong shape (an empty successful-looking tile, or ignored lines). Do not pass explicit-format SQL to runReadInto({ format: 'Table' }).

• Recommended: Dashboard Panel queries reject any explicit FORMAT with a clear tile error ("Dashboard panels require structured streaming results. Remove the explicit FORMAT clause.") — a small, deliberate behavior change that avoids silent corruption.
• Compatibility (more work): preserve explicit FORMAT JSON through a dedicated normalization/raw path, still rejecting other formats — weakens the one-path goal.

6. Streamed-result → tile adapter

Pin the conversion so applyTileResult/tileFooter keep their shape:

function dashboardTileResult(result, startedAt, finishedAt) {
  return {
    columns: result.columns, rows: result.rows,
    error: result.error, cancelled: result.cancelled,
    meta: {
      rows: result.rows.length,
      ms: Math.round(finishedAt - startedAt),   // wall-clock, like run()'s finally
      bytes: result.progress.bytes,
      truncated: result.capped,
    },
  };
}

Only a successful, non-cancelled, current-generation result is applied and records recent values.

7. Concurrency scope — make it explicit

Today full Refresh uses the 6-way runPool; runAffected() uses a direct Promise.all (unbounded). Prefer routing both through the same 6-way pool (documented as a small resource-safety improvement) rather than leaving the divergence implicit. If preserving the split, say so explicitly.

8. Remove the old path after migration

Once every tile uses runReadInto, delete the now-unused parallel machinery so future cap/settings fixes can't apply to only one path: app.runTile, queryDashboardTile, dashboardTileSql, parseJsonResult, their imports, and their tests.

---

Required tests

• Server limit is DASH_TILE_ROW_CAP + 1; client result limit is DASH_TILE_ROW_CAP.
• Exactly-cap results are not marked truncated; cap-plus-one results are trimmed and marked truncated.
• readonly:2, max_result_bytes, and param_* reach runReadInto.
• Full Refresh performs one token preflight; affected-filter waves perform one token preflight; a failed preflight issues no requests and drives sign-out.
• A new wave aborts the previous slot request before gating or queueing; a queued older wave cannot start after a newer affected wave (generation reserved at creation).
• Stale, aborted, failed, and cancelled requests neither render nor record recents.
• Streaming chunks update only loading progress; panel classification occurs once.
• Six-request full-refresh concurrency remains enforced (and affected waves, per the chosen scope).
• Explicit FORMAT follows the selected policy and never produces a silent empty result.
• Text / queryless panels issue no request.
• state.running, active-tab result, history, and workbench cancellation state remain untouched.

Non-goals

• Any behavior change to the workbench or the detached view (already on the seam).
• Progressive per-chunk chart/table rendering in tiles (separate, perf-tested feature).

---

The runReadInto seam already exists (from #185), so this is a focused, well-scoped change. Design review notes above folded in.