Skip to content
Merged
16 changes: 11 additions & 5 deletions .wiki/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,17 @@ module mocking.
## Query path

1. The editor/controller prepares SQL and typed parameters.
2. `src/net/ch-client.js` sends the HTTP request with injected auth/fetch context,
delegating generic request construction and stream mechanics through a narrow
transport contract (`src/net/clickhouse-transport.types.js` +
`src/net/clickhouse-http-transport.js`, #585 Phase 1) — auth/epoch/retry policy
stays in `ch-client.js`.
2. `src/net/ch-client.js`'s exported `queryJson`/`runQuery`/`exportQuery` send the
HTTP request through `src/net/authenticated-clickhouse-request.js` (#630
Phase 6), which owns auth/epoch/retry/lifecycle policy (moved out of
`ch-client.js`'s former `authedFetch`/`transportFor(ctx)`, deleted outright)
and builds the `@altinity/clickhouse-http` package client directly, composing
it with the package's response consumers; the callers keep their own
product-level result/error handling. The narrow transport contract
(`src/net/clickhouse-transport.types.js` + `src/net/clickhouse-http-transport.js`,
#585 Phase 1) is no longer the ordinary path — it now remains only as the
frozen-lease `killQueryWithLease` bypass's compatibility route, through
Phase 6; Phase 7 is expected to retire it.
3. `JSONStringsEachRowWithProgress` is folded line by line by pure stream logic.
4. Results resolve through the panel registry to table, chart, logs, KPI, filter,
text, or graph-oriented renderers.
Expand Down
45 changes: 40 additions & 5 deletions .wiki/Decisions-and-Roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,11 +244,46 @@ Two roadmap tracks are current:
underlying generic mechanics changed owner, and the existing parser/
helper bodies moved rather than being redesigned. This required revising
the architecture boundary itself (see below) since SQL Browser language
consumers now legitimately import the package outside `src/net/**`. Still
deferred to later phases: an authentication-composition rewrite (Phase 6),
and `runQuery`/`exportQuery`/the remaining request transport seam's own
eventual migration/deletion plus the Phase-4 consuming query APIs' actual
cutover (Phase 7). See
consumers now legitimately import the package outside `src/net/**`.

**Phase 6** (merged) composes SQL Browser authentication through one
new module, `src/net/authenticated-clickhouse-request.ts` — a real
move+delete of the normal-request auth/epoch/refresh/lifecycle policy
that used to live in `ch-client.ts` as `authedFetch()`/a module-private
`transportFor(ctx)`: both are gone, with no forwarding alias, no second
retry loop, and no second Authorization constructor. The new module
builds the package client directly
(`createClickHouseHttpClient(...).request()`) rather than through the
compatibility transport adapter, and exposes `authenticatedRequest()`
(the moved trust-boundary loop) plus `authenticatedJson()`/
`authenticatedText()`/`authenticatedProgress()`, each composing it with
exactly one matching package response consumer. `ChCtx` now `extends`
the new module's narrower `AuthenticatedRequestCtx` instead of
redeclaring its fields, adding only `dataLakeCatalogSettingUnsupported`.
`queryJson()` is the first real production consumer of the package's
JSON response consumer, translating the package's `ClickHouseError`
back to `queryJson`'s existing plain-`Error` compatibility shape (same
parsed message); `runQuery()`/`exportQuery()` switch only their
`authedFetch()` call to the new raw `authenticatedRequest()` entrypoint,
keeping their own result/error/body handling unchanged.
`killQueryWithLease()`'s frozen-lease bypass is untouched — it already
built its own one-shot transport directly from the frozen lease, never
through `ChCtx`, so it does not route through the new mutable-context
auth loop. `build/check-boundaries.mjs`'s two existing #585
transport-leaf forbidden lists and the #512 `connectionAuthorityFiles`
lifecycle-authority list now name the new module too (a data extension
of existing rules, not a new scanner); `ch-client.ts` stays in those
lists through Phase 7. Real-browser coverage: authenticated-path
variants of the existing post-header cancellation scenarios 5-9
(`tests/e2e/clickhouse-http-transport.{html,spec.js}`), proving the
identical native Fetch/Response/cancellation semantics survive being
driven through a real, production-shaped `AuthenticatedRequestCtx`
(synthetic test credentials, one deterministic epoch) in both Chromium
and WebKit. Still deferred to **Phase 7**: `runQuery`/`exportQuery`'s
cutover onto the package's convenience consuming query APIs and their
own result/export ownership migration, the remaining
`killQuery`/`killQueryWithLease` transport migration, and deletion of
the now-superseded transport-adapter compatibility seam. See
[[Source-Map]] and [[Architecture]] for the file-level detail and
`build/check-boundaries.mjs`'s Rules A–D plus the Phase 3/5 narrow
legacy-owner rules for the mechanical boundary enforcement: package↔root-src
Expand Down
9 changes: 5 additions & 4 deletions .wiki/Source-Map.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,11 @@ Back to [[Home]]. Related: [[Architecture]], [[Product-and-Features]].
| `src/dashboard/application/dashboard-repaint-plan.js` | pure repaint-decision arbitration extracted from `ui/dashboard.js`'s `renderDashboard` effect (#589) |
| `src/ui/dashboard-tile-gestures.js` | Dashboard corner-drag resize, Command/Ctrl-drag reorder, and modifier-cue controller, extracted from `ui/dashboard.js` behind an injected `TileGestureDeps` seam (#589) |
| `src/state.js` | signals-backed state model and persistence operations |
| `src/net/ch-client.js` | ClickHouse HTTP execution and schema calls; auth/epoch/retry policy, product operations, `ChCtx` (#585 Phase 1: generic request/stream mechanics delegate through the transport seam below; #630 Phase 2: `chUrl` re-exported from `@altinity/clickhouse-http`; #630 Phase 3: `streamLines` called directly, `parseExceptionText`/`findExceptionFrame`/`StreamLine`/`StreamCallbacks` re-exported; #630 Phase 4: unaffected — the package's new consuming query APIs/`killQuery` are additive and not yet consumed here; #630 Phase 5: `sqlString` also imported directly from the package, replacing the retired `../core/format.js` import) |
| `src/net/clickhouse-transport.types.js` | Type-only `ClickHouseTransport` contract — `send()` ONLY since #630 Phase 3 (`streamLines`/`StreamCallbacks` moved to the package); `TransportDeps`/`TransportRequest` alias the package's own types (#585 Phase 1; #630 Phase 2) |
| `src/net/clickhouse-http-transport.js` | `createHttpTransport` — temporary compatibility adapter, REQUEST/SEND-ONLY since #630 Phase 3: `send()` delegates to `@altinity/clickhouse-http`'s `request()`; no stream member at all (`ch-client.ts`'s `runQuery` calls the package's `streamLines` directly instead) (#585 Phase 1; #630 Phases 2-3) |
| `packages/clickhouse-http/src/` | First-party npm workspace package (repo's first) — `url.ts` (`chUrl`, the ONE URL-serializer implementation), `client.ts` (`createClickHouseHttpClient`, the low-level request/Fetch invocation, plus #630 Phase 4's `queryJson`/`queryText`/`queryProgress` convenience methods and stateless `killQuery` — since #630 Phase 5, `killQuery` quotes through this package's own `sql-quote.ts` `sqlString`, and the Phase-4 private `quoteKillQueryId` stopgap is gone), `progress-stream.ts` (`streamLines`, the ONE progress-bearing JSON-lines read loop, plus the canonical `StreamLine`/`StreamCallbacks`/`ProgressMetaColumn` wire types), `exceptions.ts` (`parseExceptionText`, `findExceptionFrame`/`ExceptionFrame` — byte-oriented, no caller-side latin1 conversion — plus #630 Phase 4's minimal `ClickHouseError`), `response.ts` (#630 Phase 4, new — `ensureClickHouseSuccess`, `consumeJsonResponse`/`consumeTextResponse`/`consumeProgressResponse`), and — new in #630 Phase 5 — `sql-quote.ts` (`sqlString`/`quoteIdent`/`qualifyIdent`, the ONE ClickHouse SQL-quoting implementation, moved verbatim from `src/core/format.ts`), `clickhouse-type.ts` (`parseClickHouseType`/`analyzeTypeModifiers`/`canonicalType`/the wrapper+enum helpers, the ONE generic type-expression grammar, moved verbatim from `src/core/clickhouse-type.ts` minus SQL Browser's `isSupportedOptionScalar` policy, which stayed at `src/core/param-type.ts`), `sql-spans.ts` (`scanSpans`/`Span`/`SpanKind`, the ONE shared lexical scanner, re-exported because surviving SQL Browser SQL-analysis modules still need it, moved verbatim from `src/core/sql-spans.ts`), and package-private `quoted-span.ts` (`scanDelimited`, moved verbatim from `src/core/quoted-span.ts`, not re-exported) — public export only, zero runtime dependencies, zero bare-specifier imports, no SQL Browser `src/**` dependency (#630 Phase 2; progress-stream/exceptions since Phase 3; response/query/kill APIs since Phase 4 — additive, not yet consumed by any `src/**` caller; SQL quoting/type grammar/scanner since Phase 5 — real production consumers retargeted). Bare package access is now two categories: transport/protocol APIs stay `src/net/**`-only; the pure-language exports above (quoting, type grammar, scanner) may be imported by their real SQL Browser consumers anywhere outside `src/net/**` too (mechanically allowlisted, `build/check-boundaries.mjs` Rule D) |
| `src/net/ch-client.js` | ClickHouse HTTP execution and schema calls; product operations, `ChCtx` (#585 Phase 1: generic request/stream mechanics delegate through the transport seam below; #630 Phase 2: `chUrl` re-exported from `@altinity/clickhouse-http`; #630 Phase 3: `streamLines` called directly, `parseExceptionText`/`findExceptionFrame`/`StreamLine`/`StreamCallbacks` re-exported; #630 Phase 4: unaffected — the package's new consuming query APIs/`killQuery` are additive and not yet consumed here; #630 Phase 5: `sqlString` also imported directly from the package, replacing the retired `../core/format.js` import; #630 Phase 6: auth/epoch/retry/lifecycle policy (`authedFetch`/`transportFor(ctx)`) MOVED to `authenticated-clickhouse-request.js` below — `ch-client.js` is now the product/query/export COMPATIBILITY owner: `ChCtx` `extends AuthenticatedRequestCtx` and adds only `dataLakeCatalogSettingUnsupported`; `queryJson()` delegates to `authenticatedJson()` with a `ClickHouseError`→`Error` compatibility translation; `runQuery`/`exportQuery` call the new module's raw `authenticatedRequest()`, keeping their own result/error/body handling; `killQueryWithLease`'s frozen-lease bypass is untouched) |
| `src/net/authenticated-clickhouse-request.js` | **New in #630 Phase 6.** The sole normal-request auth/epoch/refresh/lifecycle owner: `authenticatedRequest()` (the moved `authedFetch` trust-boundary loop, now building the package's `createClickHouseHttpClient(...).request()` directly instead of going through the compatibility transport) plus `authenticatedJson()`/`authenticatedText()`/`authenticatedProgress()`, each composing it with exactly one matching package response consumer (`consumeJsonResponse`/`consumeTextResponse`/`consumeProgressResponse`). Declares the narrow `AuthenticatedRequestCtx` seam `ch-client.js`'s `ChCtx` now extends. Named in `build/check-boundaries.mjs`'s #585 transport-leaf forbidden lists and the #512 `connectionAuthorityFiles` lifecycle-authority list |
| `src/net/clickhouse-transport.types.js` | Type-only `ClickHouseTransport` contract — `send()` ONLY since #630 Phase 3 (`streamLines`/`StreamCallbacks` moved to the package); `TransportDeps`/`TransportRequest` alias the package's own types (#585 Phase 1; #630 Phase 2). Since #630 Phase 6, its one remaining production caller is `killQueryWithLease`'s frozen-lease bypass — the normal-request path moved to `authenticated-clickhouse-request.js`, which builds the package client directly |
| `src/net/clickhouse-http-transport.js` | `createHttpTransport` — temporary compatibility adapter, REQUEST/SEND-ONLY since #630 Phase 3: `send()` delegates to `@altinity/clickhouse-http`'s `request()`; no stream member at all (`ch-client.ts`'s `runQuery` calls the package's `streamLines` directly instead) (#585 Phase 1; #630 Phases 2-3). Since #630 Phase 6, its one remaining production caller is `killQueryWithLease` |
| `packages/clickhouse-http/src/` | First-party npm workspace package (repo's first) — `url.ts` (`chUrl`, the ONE URL-serializer implementation), `client.ts` (`createClickHouseHttpClient`, the low-level request/Fetch invocation, plus #630 Phase 4's `queryJson`/`queryText`/`queryProgress` convenience methods and stateless `killQuery` — since #630 Phase 5, `killQuery` quotes through this package's own `sql-quote.ts` `sqlString`, and the Phase-4 private `quoteKillQueryId` stopgap is gone), `progress-stream.ts` (`streamLines`, the ONE progress-bearing JSON-lines read loop, plus the canonical `StreamLine`/`StreamCallbacks`/`ProgressMetaColumn` wire types), `exceptions.ts` (`parseExceptionText`, `findExceptionFrame`/`ExceptionFrame` — byte-oriented, no caller-side latin1 conversion — plus #630 Phase 4's minimal `ClickHouseError`), `response.ts` (#630 Phase 4, new — `ensureClickHouseSuccess`, `consumeJsonResponse`/`consumeTextResponse`/`consumeProgressResponse`), and — new in #630 Phase 5 — `sql-quote.ts` (`sqlString`/`quoteIdent`/`qualifyIdent`, the ONE ClickHouse SQL-quoting implementation, moved verbatim from `src/core/format.ts`), `clickhouse-type.ts` (`parseClickHouseType`/`analyzeTypeModifiers`/`canonicalType`/the wrapper+enum helpers, the ONE generic type-expression grammar, moved verbatim from `src/core/clickhouse-type.ts` minus SQL Browser's `isSupportedOptionScalar` policy, which stayed at `src/core/param-type.ts`), `sql-spans.ts` (`scanSpans`/`Span`/`SpanKind`, the ONE shared lexical scanner, re-exported because surviving SQL Browser SQL-analysis modules still need it, moved verbatim from `src/core/sql-spans.ts`), and package-private `quoted-span.ts` (`scanDelimited`, moved verbatim from `src/core/quoted-span.ts`, not re-exported) — public export only, zero runtime dependencies, zero bare-specifier imports, no SQL Browser `src/**` dependency (#630 Phase 2; progress-stream/exceptions since Phase 3; response/query/kill APIs since Phase 4 — additive, not consumed by any `src/**` caller until Phase 6; SQL quoting/type grammar/scanner since Phase 5 — real production consumers retargeted). Since #630 Phase 6, `src/net/authenticated-clickhouse-request.js` is the first real `src/**` consumer of `request()` plus the non-consuming classifier/JSON/text/progress consumers — the convenience `queryJson`/`queryText`/`queryProgress` client methods themselves still have no `src/**` consumer (Phase 7). Bare package access is now two categories: transport/protocol APIs stay `src/net/**`-only; the pure-language exports above (quoting, type grammar, scanner) may be imported by their real SQL Browser consumers anywhere outside `src/net/**` too (mechanically allowlisted, `build/check-boundaries.mjs` Rule D) |
| `src/net/oauth.js` | OAuth flow/token exchange |
| `src/editor/editor-port.js` | SQL editor contract and safe no-op port |
| `src/editor/codemirror-adapter.js` | SQL CodeMirror 6 adapter |
Expand Down
65 changes: 63 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,67 @@ auto-generated per-PR notes; this file is the curated, human-readable history.
## [Unreleased]

### Added
- **#630 Phase 6: compose SQL Browser authentication through one
`authenticated-clickhouse-request.ts` layer over the package's
`request()` and response consumers.** The normal-request auth/epoch/
refresh/lifecycle policy that used to live in `src/net/ch-client.ts` as
`authedFetch()`/a module-private `transportFor(ctx)` moves to a new
module, `src/net/authenticated-clickhouse-request.ts` — a real move+
delete, not an additive layer: both are gone from `ch-client.ts`, with
no forwarding alias, no second retry loop, and no second Authorization
constructor. The new module builds the `@altinity/clickhouse-http`
package client directly (`createClickHouseHttpClient(...).request()`)
instead of going through the compatibility transport adapter, and
exposes `authenticatedRequest()` (the moved trust-boundary loop) plus
`authenticatedJson()`/`authenticatedText()`/`authenticatedProgress()`,
each composing it with exactly one matching package response consumer
(`consumeJsonResponse`/`consumeTextResponse`/`consumeProgressResponse`).
`ChCtx` now `extends` the new module's narrower `AuthenticatedRequestCtx`
instead of redeclaring its fields, adding only
`dataLakeCatalogSettingUnsupported` — the one field genuinely specific
to the product client. `AuthenticatedCancellationLease` stays exported
from `ch-client.ts`, and `killQueryWithLease`'s frozen-lease bypass is
untouched: it already built its own one-shot transport directly from
the frozen lease, never through `ChCtx`, so it does not route through
the new mutable-context auth loop (hard invariant 8/13).

`queryJson()` is the first real production consumer of the package's
response-consumer layer: it now delegates to `authenticatedJson()`,
translating the package's `ClickHouseError` back to `queryJson`'s
EXISTING plain-`Error` compatibility shape (same parsed message) so this
phase adopts the new consumer without changing an existing SQL Browser
API. `runQuery()`/`exportQuery()` switch only their `authedFetch()` call
to the new raw `authenticatedRequest()` entrypoint, keeping their own
Table/KPI/raw format mapping, row-cap settings, non-2xx parsing, and
streaming exactly as before — their full package-consumer/result/export
cutover remains Phase 7, as does `authenticatedText()`/
`authenticatedProgress()`'s adoption by any other caller.

`build/check-boundaries.mjs`'s two existing #585 transport-leaf
forbidden lists (`clickhouse-http-transport.ts`,
`clickhouse-transport.types.ts`) and the #512 `connectionAuthorityFiles`
lifecycle-authority list now name the new module as the current auth/
lifecycle owner they must not reach/regain — a data extension of
existing rules, not a new scanner. `ch-client.ts` stays in the
transport-leaf forbidden lists too through Phase 7.

Real-browser coverage: `tests/e2e/clickhouse-http-transport.{html,spec.js}`
gains authenticated-path variants of the existing post-header
cancellation scenarios 5-9, driving `authenticatedRequest()`/
`authenticatedProgress()` through a real, production-shaped
`AuthenticatedRequestCtx` (synthetic test credentials, one deterministic
epoch) against the same real cross-origin fault server — proving the
identical native Fetch/Response/cancellation semantics survive SQL
Browser's own credential/epoch composition, in both Chromium and WebKit,
not just the compatibility transport/package client with an
already-resolved Authorization.

Only A12 (one authenticated request owner over the package) and A13
(epoch/refresh/lifecycle/cancellation invariants remain regression-
tested and unchanged) are newly claimed; A14-A18 (the remaining
`runQuery`/`exportQuery`/transport-seam migration and deletion) stay
deferred to Phase 7.

- **#630 Phase 5: move ClickHouse SQL quoting and generic type-expression
grammar into `@altinity/clickhouse-http`.** `sqlString`, `quoteIdent`, and
`qualifyIdent` now have one package implementation (`sql-quote.ts`),
Expand Down Expand Up @@ -58,8 +119,8 @@ auto-generated per-PR notes; this file is the curated, human-readable history.
`clickhouse-http-sql-spans.test.ts`, `clickhouse-http-sql-quote.test.ts`);
the moved `isSupportedOptionScalar` describe block now lives in
`tests/unit/param-type.test.ts` alongside its relocated implementation.
Phase 6 auth composition and Phase 7 query/export/transport-seam cutover
remain deferred.
Phase 6 auth composition landed next (see above); Phase 7's
query/export/transport-seam cutover remains deferred.

- **#630 Phase 4: add consuming query APIs, a minimal ClickHouse HTTP error,
and a stateless `KILL QUERY` to `@altinity/clickhouse-http`.** Purely
Expand Down
Loading