From f723634e796b3a7a0332639ec0201871284b30c8 Mon Sep 17 00:00:00 2001 From: Boris Tyshkevich Date: Mon, 27 Jul 2026 17:20:50 +0200 Subject: [PATCH 1/2] refactor(#447): move shop-charts.json and ontime-charts.json to Dashboard v2 Completes the phase-3 example rewrite spec-examples.test.js's pinned-version test had been waiting on: shop-charts.json and ontime-charts.json move to portable-bundle v2 / Dashboard document v2, dropping their curated `filters` arrays in favor of inferred Variables, matching clickhouse-operations.json's earlier migration (#458 follow-up). Audited every from/to/country/category/carrier/origin filter's `targets` list against the {name:Type} placeholders actually present in each tile's SQL: they already matched exactly, so removing the curated model changes no runtime variable-to-query assignment. build-ontime-charts.mjs now threads its bundle's own `version` through to writeExampleBundle instead of silently reverting to v1 on regeneration. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011sMwR4BHM85MExSBYXzTYj --- CHANGELOG.md | 13 +++++ examples/mjs/README.md | 27 ++++++++--- examples/mjs/build-ontime-charts.mjs | 5 +- examples/mjs/normalize-examples.mjs | 4 +- examples/ontime-charts.json | 72 ++-------------------------- examples/shop-charts.json | 68 ++------------------------ tests/unit/spec-examples.test.js | 12 +++-- 7 files changed, 52 insertions(+), 149 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bc159d3c..2f9961b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,19 @@ auto-generated per-PR notes; this file is the curated, human-readable history. ## [Unreleased] +### Changed +- **`shop-charts.json` and `ontime-charts.json` moved to portable-bundle v2 / + Dashboard document v2**, dropping their curated `filters` arrays in favor of + inferred Variables — the phase-3 example rewrite `spec-examples.test.js` had + been pinning them against, following `clickhouse-operations.json`'s earlier + move (#458 follow-up). Every `from`/`to`/`country`/`category`/`carrier`/ + `origin` variable was already targeting exactly the tiles whose query + declares its `{name:Type}` placeholder, so the curated `targets` lists these + examples carried added nothing the placeholders didn't already say; removing + them changes no runtime behavior. `build-ontime-charts.mjs` now passes its + bundle's own `version` through to `writeExampleBundle` instead of silently + reverting to v1 on its next regeneration. + ### Added - **Closing a dirty tab, or leaving the page, now confirms first** (#466). The tab strip's close button asks before discarding an unsaved draft — a normal diff --git a/examples/mjs/README.md b/examples/mjs/README.md index 43f41c15..5af1a843 100644 --- a/examples/mjs/README.md +++ b/examples/mjs/README.md @@ -1,10 +1,22 @@ # Example Bundles and Generators -The checked-in JSON files under `examples/` are canonical **portable bundle -v1** documents. Query definitions use saved-query **Spec v1** and every -Dashboard example includes an explicit **Dashboard document v1** with semantic -tile order, filter definitions, and either `flow@1` or `grafana-grid@1` layout. -Every grid layout carries a complete `flow@1` fallback. +The checked-in JSON files under `examples/` are canonical **portable bundle** +documents. Query definitions use saved-query **Spec v1**, and every Dashboard +example includes an explicit **Dashboard document** with semantic tile order +and either `flow@1` or `grafana-grid@1` layout. Every grid layout carries a +complete `flow@1` fallback. + +The three flagship, hand-authored dashboards — `clickhouse-operations.json`, +`shop-charts.json`, and `ontime-charts.json` — are on **portable bundle v2 / +Dashboard document v2** (#447/#459): there is no curated `filters` array. +Dashboard variables are inferred from the `{name:Type}` placeholders in the +tiled queries' own SQL, matched by exact name, so a variable applies to +exactly the tiles whose query declares it — adding, removing, or renaming a +placeholder is the only way to change a variable's assignment. The generated +Iceberg examples (`iceberg-catalog-dashboard.json`, `iceberg-dba-dashboard.json`, +and the drill-down template) still go through `buildDashboard()` and stay on +**v1** with a minimal curated `filters` array (one per SQL parameter, no +label/default/targets) until that generator itself becomes version-aware. Legacy Library v1/v2 JSON remains importable for compatibility, but it is not an authoring format for new or regenerated examples. @@ -20,8 +32,9 @@ authoring format for new or regenerated examples. ## Generators - `build-ontime-charts.mjs` refreshes the live panel schema keys in - `ontime-charts.json` while preserving its authored grid, filters, KPI - configuration, tile order, and flow fallback. + `ontime-charts.json` while preserving its authored grid, KPI configuration, + tile order, and flow fallback. It does not touch Dashboard variables — those + are inferred at runtime from the query placeholders it leaves untouched. - `build-iceberg-install.mjs` regenerates `iceberg-install.json`. - `build-iceberg-dashboards.mjs` regenerates `iceberg-catalog-dashboard.json` and `iceberg-dba-dashboard.json`. diff --git a/examples/mjs/build-ontime-charts.mjs b/examples/mjs/build-ontime-charts.mjs index cb0de77d..ffeb3d86 100644 --- a/examples/mjs/build-ontime-charts.mjs +++ b/examples/mjs/build-ontime-charts.mjs @@ -1,7 +1,9 @@ // Refresh the live schema keys in examples/ontime-charts.json against the // configured antalya ClickHouse connection. The checked-in bundle is the // authored source of truth for SQL, semantic tile order, grafana-grid sizing, -// filters/defaults/targets, KPI field configuration, and the flow fallback. +// KPI field configuration, and the flow fallback. Dashboard variables are not +// touched here — they are inferred at runtime from the `{name:Type}` +// placeholders already in the SQL this script leaves untouched. // // Run: node examples/mjs/build-ontime-charts.mjs @@ -47,5 +49,6 @@ writeExampleBundle(outPath, { metadata: document.metadata, queries: document.queries, dashboards: document.dashboards, + version: document.version, }); console.log(`wrote ${outPath}`); diff --git a/examples/mjs/normalize-examples.mjs b/examples/mjs/normalize-examples.mjs index b67bf36c..1d84f14d 100644 --- a/examples/mjs/normalize-examples.mjs +++ b/examples/mjs/normalize-examples.mjs @@ -20,7 +20,7 @@ const checkOnly = process.argv.includes('--check'); const CONFIG = { 'shop-charts.json': { - id: 'shop-analytics', title: 'Shop analytics', authoredDashboard: true, + id: 'shop-analytics', title: 'Shop analytics', authoredDashboard: true, version: 2, description: 'Revenue, buyers, products, geography, and traffic over the shop-demo.sql dataset.', }, 'clickhouse-operations.json': { @@ -28,7 +28,7 @@ const CONFIG = { description: 'Operator-first server overview, resources, background work, and investigation views.', }, 'ontime-charts.json': { - id: 'ontime-flights', title: 'On-time flights', authoredDashboard: true, + id: 'ontime-flights', title: 'On-time flights', authoredDashboard: true, version: 2, description: 'Flight punctuality, volume, carriers, airports, delays, and cancellations with a shared 2023 slice.', }, 'iceberg-catalog-dashboard.json': { diff --git a/examples/ontime-charts.json b/examples/ontime-charts.json index 0d8fc760..a0915b9f 100644 --- a/examples/ontime-charts.json +++ b/examples/ontime-charts.json @@ -1,7 +1,7 @@ { - "$schema": "https://altinity.com/schemas/altinity-sql-browser/portable-bundle-v1.schema.json", + "$schema": "https://altinity.com/schemas/altinity-sql-browser/portable-bundle-v2.schema.json", "format": "altinity-sql-browser/portable-bundle", - "version": 1, + "version": 2, "exportedAt": "2026-07-22T00:00:00.000Z", "metadata": { "name": "On-time flights", @@ -360,7 +360,7 @@ ], "dashboards": [ { - "documentVersion": 1, + "documentVersion": 2, "id": "ontime-flights", "title": "On-time flights", "description": "Flight punctuality, volume, carriers, airports, delays, and cancellations with a shared 2023 slice.", @@ -434,72 +434,6 @@ } } }, - "filters": [ - { - "id": "ontime-date-from", - "parameter": "from", - "label": "From", - "defaultValue": "2023-01-01", - "defaultActive": true, - "targets": [ - "tile-ontime-kpis", - "tile-s3", - "tile-s4", - "tile-s1", - "tile-s6", - "tile-s7", - "tile-s5" - ] - }, - { - "id": "ontime-date-to", - "parameter": "to", - "label": "To", - "defaultValue": "2023-12-31", - "defaultActive": true, - "targets": [ - "tile-ontime-kpis", - "tile-s3", - "tile-s4", - "tile-s1", - "tile-s6", - "tile-s7", - "tile-s5" - ] - }, - { - "id": "ontime-carrier", - "parameter": "carrier", - "label": "Carrier", - "defaultValue": [], - "defaultActive": false, - "targets": [ - "tile-ontime-kpis", - "tile-s3", - "tile-s4", - "tile-s1", - "tile-s6", - "tile-s7", - "tile-s5" - ] - }, - { - "id": "ontime-origin", - "parameter": "origin", - "label": "Origin airport", - "defaultValue": [], - "defaultActive": false, - "targets": [ - "tile-ontime-kpis", - "tile-s3", - "tile-s4", - "tile-s1", - "tile-s6", - "tile-s7", - "tile-s5" - ] - } - ], "tiles": [ { "id": "tile-ontime-kpis", diff --git a/examples/shop-charts.json b/examples/shop-charts.json index df2fd8a5..914bf2c1 100644 --- a/examples/shop-charts.json +++ b/examples/shop-charts.json @@ -1,7 +1,7 @@ { - "$schema": "https://altinity.com/schemas/altinity-sql-browser/portable-bundle-v1.schema.json", + "$schema": "https://altinity.com/schemas/altinity-sql-browser/portable-bundle-v2.schema.json", "format": "altinity-sql-browser/portable-bundle", - "version": 1, + "version": 2, "exportedAt": "2026-07-22T00:00:00.000Z", "metadata": { "name": "Shop analytics", @@ -259,7 +259,7 @@ ], "dashboards": [ { - "documentVersion": 1, + "documentVersion": 2, "id": "shop-analytics", "title": "Shop analytics", "description": "Revenue, buyers, products, geography, and traffic over the shop-demo.sql dataset.", @@ -333,68 +333,6 @@ } } }, - "filters": [ - { - "id": "shop-from", - "parameter": "from", - "label": "From", - "defaultValue": "-90d", - "defaultActive": true, - "targets": [ - "tile-shop-kpis", - "tile-shop-daily-revenue", - "tile-shop-revenue-by-country", - "tile-shop-revenue-by-category", - "tile-shop-top-products", - "tile-shop-daily-active-users", - "tile-shop-traffic-by-hour" - ] - }, - { - "id": "shop-to", - "parameter": "to", - "label": "To", - "defaultValue": "now", - "defaultActive": true, - "targets": [ - "tile-shop-kpis", - "tile-shop-daily-revenue", - "tile-shop-revenue-by-country", - "tile-shop-revenue-by-category", - "tile-shop-top-products", - "tile-shop-daily-active-users", - "tile-shop-traffic-by-hour" - ] - }, - { - "id": "shop-country", - "parameter": "country", - "label": "Country", - "defaultValue": [], - "defaultActive": false, - "targets": [ - "tile-shop-kpis", - "tile-shop-daily-revenue", - "tile-shop-revenue-by-country", - "tile-shop-top-products", - "tile-shop-daily-active-users", - "tile-shop-traffic-by-hour" - ] - }, - { - "id": "shop-category", - "parameter": "category", - "label": "Category", - "defaultValue": [], - "defaultActive": false, - "targets": [ - "tile-shop-kpis", - "tile-shop-revenue-by-category", - "tile-shop-top-products", - "tile-shop-traffic-by-hour" - ] - } - ], "tiles": [ { "id": "tile-shop-kpis", diff --git a/tests/unit/spec-examples.test.js b/tests/unit/spec-examples.test.js index dfd03de5..4e905d4a 100644 --- a/tests/unit/spec-examples.test.js +++ b/tests/unit/spec-examples.test.js @@ -57,11 +57,13 @@ describe('schema artifacts and examples', () => { // every example is already covered by the "migrates every shipped example // bundle" test above; this test is specifically about the shape the files // are still committed in. - // clickhouse-operations.json moved to portable-bundle v2 / Dashboard v2 ahead - // of the other checked-in examples (#458 follow-up): its authored Dashboard - // dropped the curated `filters` array in favor of inferred Variables. The - // rest stay pinned to v1 until phase 3's broader example rewrite. - const V2_EXAMPLES = new Set(['clickhouse-operations.json']); + // All three authored, checked-in Dashboards (clickhouse-operations first, + // #458 follow-up; shop-charts and ontime-charts in the phase-3 rewrite) moved + // to portable-bundle v2 / Dashboard v2, dropping the curated `filters` array + // in favor of inferred Variables. The generated Iceberg examples still go + // through `buildDashboard()` (documentVersion 1, filters required) and stay + // pinned to v1 until that generator itself becomes version-aware. + const V2_EXAMPLES = new Set(['clickhouse-operations.json', 'ontime-charts.json', 'shop-charts.json']); it('keeps every checked-in JSON example on its pinned portable bundle version with explicit Dashboard documents', () => { const examples = resolve(root, 'examples'); From 196d16156801371b9d1d00ced9f7fac547ce5ecd Mon Sep 17 00:00:00 2001 From: Boris Tyshkevich Date: Mon, 27 Jul 2026 17:57:05 +0200 Subject: [PATCH 2/2] feat: restore option lists for the flagship dashboards' Array/enum variables The v2 migration left every Array/enum-shaped variable (country/category, carrier/origin, user/query_kind/exception_code/query_hash/metric/ is_initial_query) as bare freeform-text input, with no configured variableConfigs. This wasn't new breakage from the v2 move itself: #447's mechanical minimum pass had already dropped every curated filter's sourceQueryId, and the pre-#447 option-source queries (shop-filter-options, ontime-filter-options, gco-filter) returned one row of ARRAY-typed columns -- a shape the v2 option-SQL contract (one row per option, two String columns: value, then label) can't reuse directly. Port each old option-source query into its own variableConfigs[name].sql under the new contract, restoring the searchable (multi-)select these variables used to have before #447. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_011sMwR4BHM85MExSBYXzTYj --- CHANGELOG.md | 13 +++++++++++++ examples/clickhouse-operations.json | 22 +++++++++++++++++++++- examples/mjs/README.md | 8 +++++++- examples/ontime-charts.json | 10 +++++++++- examples/shop-charts.json | 10 +++++++++- 5 files changed, 59 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2f9961b2..ade2e416 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,19 @@ auto-generated per-PR notes; this file is the curated, human-readable history. them changes no runtime behavior. `build-ontime-charts.mjs` now passes its bundle's own `version` through to `writeExampleBundle` instead of silently reverting to v1 on its next regeneration. +- **The three flagship dashboards' Array/enum variables get real option lists + again.** `country`/`category` (`shop-charts.json`), `carrier`/`origin` + (`ontime-charts.json`), and `user`/`query_kind`/`exception_code`/ + `query_hash`/`metric`/`is_initial_query` (`clickhouse-operations.json`) had + been reduced to bare freeform-text Array inputs ever since #447's mechanical + minimum pass dropped every curated filter's `sourceQueryId` — the pre-#447 + option-source queries existed (`shop-filter-options`, `ontime-filter-options`, + `gco-filter`) but returned one row of ARRAY-typed columns, a shape the v2 + option-SQL contract (one row per option, two String columns: value, then + label) cannot reuse directly. Each is now ported into its own + `variableConfigs[name].sql`, so these variables render a searchable + (multi-)select again instead of pure freeform text, restoring the pre-#447 + browsing experience under the new inferred model. ### Added - **Closing a dirty tab, or leaving the page, now confirms first** (#466). The diff --git a/examples/clickhouse-operations.json b/examples/clickhouse-operations.json index 725f8f62..7e2b45fa 100644 --- a/examples/clickhouse-operations.json +++ b/examples/clickhouse-operations.json @@ -2916,7 +2916,27 @@ "id": "tile-ops-recent-server-logs", "queryId": "ops-recent-server-logs" } - ] + ], + "variableConfigs": { + "user": { + "sql": "SELECT DISTINCT initial_user, initial_user\nFROM merge(system, '^query_log')\nWHERE event_time >= now() - INTERVAL 7 DAY AND initial_user != ''\nORDER BY initial_user" + }, + "query_kind": { + "sql": "SELECT DISTINCT query_kind, query_kind\nFROM merge(system, '^query_log')\nWHERE event_time >= now() - INTERVAL 7 DAY AND query_kind != ''\nORDER BY query_kind" + }, + "exception_code": { + "sql": "SELECT toString(exception_code), concat(toString(exception_code), ' · ', errorCodeToName(exception_code))\nFROM (\n SELECT exception_code\n FROM merge(system, '^query_log')\n WHERE event_time >= now() - INTERVAL 7 DAY AND exception_code != 0\n GROUP BY exception_code\n ORDER BY count() DESC\n LIMIT 100\n)" + }, + "query_hash": { + "sql": "SELECT toString(normalized_query_hash), concat(toString(normalized_query_hash), ' · ', substring(sample_query, 1, 80))\nFROM (\n SELECT normalized_query_hash, anyHeavy(replaceAll(query, '\\n', ' ')) AS sample_query\n FROM merge(system, '^query_log')\n WHERE event_time >= now() - INTERVAL 7 DAY AND normalized_query_hash != 0\n GROUP BY normalized_query_hash\n ORDER BY count() DESC\n LIMIT 80\n)" + }, + "metric": { + "sql": "SELECT metric, metric\nFROM (\n SELECT arrayJoin(['count', 'avg_duration', 'max_duration', 'cpu_time', 'read_bytes', 'written_bytes', 'avg_written_rows', 'result_bytes', 'network_bytes', 'memory', 'max_memory', 'network_wait', 'io_time', 'io_wait', 'zk_txns', 'read_bps', 'write_bps', 'parts_inserted', 'parts_inserted_avg', 'marks_load_time', 'marks_miss_rate', 'selected_parts', 'selected_ranges', 'selected_marks', 'exceptions', 'open_files', 'external_processing_files', 'threads1', 'threads2', 'threads3']) AS metric\n)" + }, + "is_initial_query": { + "sql": "SELECT toString(v), if(v = 1, 'Yes', 'No')\nFROM (SELECT arrayJoin([1, 0]) AS v)" + } + } } ] } diff --git a/examples/mjs/README.md b/examples/mjs/README.md index 5af1a843..dde52032 100644 --- a/examples/mjs/README.md +++ b/examples/mjs/README.md @@ -12,7 +12,13 @@ Dashboard document v2** (#447/#459): there is no curated `filters` array. Dashboard variables are inferred from the `{name:Type}` placeholders in the tiled queries' own SQL, matched by exact name, so a variable applies to exactly the tiles whose query declares it — adding, removing, or renaming a -placeholder is the only way to change a variable's assignment. The generated +placeholder is the only way to change a variable's assignment. Every +Array/enum-shaped variable (`country`/`category`, `carrier`/`origin`, +`user`/`query_kind`/`exception_code`/`query_hash`/`metric`/`is_initial_query`) +carries real, dashboard-local option SQL under `variableConfigs`, ported from +the pre-#447 curated-filter option-source queries into the two-String-column +(value, then label) contract — a direct-input free-text box only remains for +`from`/`to`/`search`. The generated Iceberg examples (`iceberg-catalog-dashboard.json`, `iceberg-dba-dashboard.json`, and the drill-down template) still go through `buildDashboard()` and stay on **v1** with a minimal curated `filters` array (one per SQL parameter, no diff --git a/examples/ontime-charts.json b/examples/ontime-charts.json index a0915b9f..c432bbf0 100644 --- a/examples/ontime-charts.json +++ b/examples/ontime-charts.json @@ -463,7 +463,15 @@ "id": "tile-s5", "queryId": "s5" } - ] + ], + "variableConfigs": { + "carrier": { + "sql": "SELECT DISTINCT Carrier, Carrier FROM ontime.fact_ontime WHERE Carrier != '' ORDER BY Carrier" + }, + "origin": { + "sql": "SELECT AirportCode, any(DisplayAirportName) AS DisplayAirportName FROM ontime.dim_airports WHERE IsLatest = 1 GROUP BY AirportCode ORDER BY DisplayAirportName" + } + } } ] } diff --git a/examples/shop-charts.json b/examples/shop-charts.json index 914bf2c1..4917e7d4 100644 --- a/examples/shop-charts.json +++ b/examples/shop-charts.json @@ -362,7 +362,15 @@ "id": "tile-shop-traffic-by-hour", "queryId": "shop-traffic-by-hour" } - ] + ], + "variableConfigs": { + "country": { + "sql": "SELECT DISTINCT country, country FROM shop.events_raw ORDER BY country" + }, + "category": { + "sql": "SELECT DISTINCT category, category FROM shop.products ORDER BY category" + } + } } ] }