Saved-query format v2: separate SQL from an extensible Spec document

[BorisTyshkevich]

Problem

The current version-1 saved-query format is a flat object:

{
  "id": "log-1",
  "name": "Server logs",
  "sql": "SELECT ...",
  "favorite": true,
  "description": "...",
  "view": "panel",
  "panel": {
    "cfg": {
      "type": "logs",
      "time": "event_time",
      "msg": "message",
      "level": "level"
    }
  }
}

Panel configuration already exists in v1, but it is a top-level saved-query field.

The saved-query model now needs to grow toward a richer, Grafana-scale specification:

• dashboard roles and filter sources;
• panel options;
• transformations;
• field configuration;
• links;
• layout and refresh behavior;
• future extension fields.

Continuing to add top-level fields creates an unstable flat namespace and makes it difficult to provide one coherent JSON authoring surface.

The current import/export/merge paths also rebuild entries from known-field whitelists. That is incompatible with an extensible spec because unknown fields may be lost.

Goal

Introduce saved-query format version 2 with clear ownership:

query.id          application-managed identity
query.sql         SQL editor document
query.specVersion query-spec schema version
query.spec        user-editable advanced configuration

Canonical export shape:

{
  "format": "altinity-sql-browser/saved-queries",
  "version": 2,
  "exportedAt": "2026-07-13T05:01:01.000Z",
  "queries": [
    {
      "id": "log-1",
      "sql": "SELECT event_time, level, logger_name, message\nFROM system.text_log",
      "specVersion": 1,
      "spec": {
        "name": "Server logs in a time range",
        "description": "Shows ClickHouse server log rows.",
        "favorite": true,
        "view": "panel",
        "panel": {
          "cfg": {
            "type": "logs",
            "time": "event_time",
            "msg": "message",
            "level": "level"
          }
        },
        "dashboard": {
          "role": "panel"
        }
      }
    }
  ]
}

Ownership

Application-managed fields

These remain outside the editable Spec document:

{
  "id": "log-1",
  "sql": "SELECT ...",
  "specVersion": 1
}

Rules:

• id is stable library identity;
• sql is owned by the SQL editor;
• specVersion identifies the per-query Spec schema;
• envelope fields remain outside each query.

User-editable Spec

The workbench Spec editor receives only query.spec:

{
  "name": "Server logs in a time range",
  "description": "Shows ClickHouse server log rows.",
  "favorite": true,
  "view": "panel",
  "panel": {
    "cfg": {
      "type": "logs",
      "time": "event_time",
      "msg": "message",
      "level": "level"
    }
  },
  "dashboard": {
    "role": "panel"
  }
}

It must not expose:

• id;
• sql;
• specVersion;
• export-envelope fields;
• legacy compatibility mirrors such as chart;
• transient runtime state.

Stable Spec namespaces

Initial top-level fields:

{
  "name": "...",
  "description": "...",
  "favorite": true,
  "view": "panel",
  "panel": {},
  "dashboard": {}
}

Reserved growth model:

{
  "panel": {
    "cfg": {},
    "fieldConfig": {
      "defaults": {},
      "overrides": []
    },
    "transformations": [],
    "links": []
  },
  "dashboard": {
    "role": "panel",
    "layout": {},
    "refresh": {}
  }
}

Only implemented fields require semantic validation.

Unknown fields must be retained.

Versioning

Envelope version

version = 2

This identifies the import/export document structure.

Spec version

specVersion = 1

This identifies the semantics of query.spec.

The versions evolve independently.

V1 migration

Every current flat v1 query upgrades into the v2 shape.

Example v1:

{
  "id": "log-1",
  "name": "Server logs",
  "sql": "SELECT ...",
  "favorite": true,
  "description": "...",
  "view": "panel",
  "panel": {
    "cfg": {
      "type": "logs"
    }
  },
  "chart": {
    "cfg": {
      "type": "line"
    }
  }
}

Upgraded v2:

{
  "id": "log-1",
  "sql": "SELECT ...",
  "specVersion": 1,
  "spec": {
    "name": "Server logs",
    "favorite": true,
    "description": "...",
    "view": "panel",
    "panel": {
      "cfg": {
        "type": "logs"
      }
    }
  }
}

Move these existing v1 fields into spec:

name
description
favorite
view
panel
future dashboard metadata when present

Keep these outside:

id
sql
specVersion

Migration rules:

• existing panel migration precedence remains authoritative;
• legacy chart is compatibility input only;
• chart is not copied into spec;
• missing name becomes Untitled;
• missing favorite becomes false;
• omitted optional fields remain omitted;
• SQL-less Text panels remain valid.

Unknown-field preservation

This is a hard invariant.

For every supported operation:

clone complete Spec
validate known fields
patch known fields
preserve unknown fields
persist complete Spec

Unknown fields inside spec must survive:

• localStorage startup;
• Save;
• Name/Description quick edit;
• favorite toggle;
• direct Panel control edits;
• dashboard-role edits;
• JSON export;
• JSON import;
• append/replace/merge;
• share encode/decode;
• tab restore.

Do not reconstruct spec from a whitelist.

Canonical internal model

Use one live query shape:

{
  id,
  sql,
  specVersion,
  spec,
}

Do not keep parallel flat and nested representations.

Use read helpers for defaults:

queryName(query)
queryDescription(query)
queryFavorite(query)
queryView(query)
queryPanel(query)
queryDashboard(query)

Use immutable update helpers:

withQuerySpec(query, nextSpec)
patchQuerySpec(query, patch)

These helpers must preserve unknown fields.

Compatibility

Import

• accept v1 and upgrade it;
• accept v2;
• reject unsupported future envelope versions;
• reject unsupported future specVersion values clearly;
• preserve unknown fields for supported versions.

Export

• write only v2;
• do not emit the legacy flat query shape;
• do not emit legacy chart;
• include specVersion on every query.

Browser storage

Upgrade on read and persist as v2 on the next write.

No destructive one-time migration is required.

Share links

Use the canonical v2 query shape or a documented v2 subset.

Do not maintain an independent flat share schema.

Structural validation

A v2 query requires:

• object query;
• non-empty string id;
• string sql;
• supported numeric specVersion;
• object spec.

Known Spec fields are validated by their owning modules.

Unknown fields are not structural errors.

Files

Expected changes:

• src/core/saved-io.js
• src/state.js
• panel compatibility helpers
• share/import/export paths
• Library query accessors
• tests across affected modules
• README format documentation
• CHANGELOG

No new runtime dependency.

Tests

V1 migration

• flat v1 query migrates into {id, sql, specVersion, spec};
• existing panel moves into spec.panel;
• chart-only legacy entries migrate correctly;
• table-over-chart precedence remains;
• Text panels with empty SQL migrate;
• existing dashboard metadata moves into spec.dashboard.

V2 round trip

• v2 export/import preserves complete Spec;
• nested unknown objects and arrays survive;
• id, sql, and specVersion stay outside Spec;
• export emits no legacy top-level authoring fields.

Mutation preservation

• favorite toggle preserves all other Spec fields;
• pencil rename/description preserves all other Spec fields;
• direct Panel edits preserve unknown fields;
• dashboard updates preserve unknown fields;
• merge-by-id preserves incoming extensions;
• duplicate detection considers complete Spec;
• share round trip preserves supported extensions.

Regression

• saved queries still open and run;
• Library search/sort reads Spec name/description and SQL;
• saved Panel/view behavior restores;
• current v1 import files remain accepted;
• dashboard favorites remain functional.

Acceptance criteria

• Canonical v2 query shape is {id, sql, specVersion, spec}.
• Existing v1 panel configuration migrates into spec.panel.
• SQL remains outside Spec.
• Identity remains outside Spec.
• Workbench Spec editor can receive only query.spec.
• Every supported v1 query upgrades without behavior loss.
• Unknown Spec fields survive all read/write paths.
• Import accepts v1 and v2.
• Export writes only v2.
• Legacy chart is migration input only.
• No parallel live flat/nested query model remains.
• Coverage gates and build pass.

Non-goals

• Workbench SQL/Spec UI.
• Dashboard Filter-source execution.
• Implementing transformations or field overrides.
• Editing id.
• Editing SQL inside Spec.
• Generic JSON Schema form generation.