[SEA-NodeJS] Pin the kernel by SHA (KERNEL_REV) + kernel-e2e CI#414
Open
msrathore-db wants to merge 1 commit into
Open
[SEA-NodeJS] Pin the kernel by SHA (KERNEL_REV) + kernel-e2e CI#414msrathore-db wants to merge 1 commit into
msrathore-db wants to merge 1 commit into
Conversation
msrathore-db
force-pushed
the
msrathore/sea-kernel-rev-pin
branch
from
c4fc625 to
56ad92f
Compare
msrathore-db
changed the base branch from
main
to
msrathore/sea-async-tls-options
msrathore-db
force-pushed
the
msrathore/sea-async-tls-options
branch
from
09fb870 to
32929aa
Compare
msrathore-db
force-pushed
the
msrathore/sea-kernel-rev-pin
branch
from
56ad92f to
93af320
Compare
vikrantpuppala
approved these changes
Jun 3, 2026
msrathore-db
force-pushed
the
msrathore/sea-kernel-rev-pin
branch
2 times, most recently
from
f87f625 to
37228bd
Compare
This was referenced Jun 3, 2026
msrathore-db
force-pushed
the
msrathore/sea-async-tls-options
branch
from
591a357 to
38eea67
Compare
The SEA napi binding is built from the kernel's private Rust source, not a published/versioned artifact, and the actual `.node` binary is gitignored — so nothing in the repo records which kernel revision the committed `native/sea/index.d.ts` / `index.js` correspond to, and the standard e2e job never builds or exercises the binding (its SEA suite skips). - `KERNEL_REV` — a single 40-char kernel commit SHA at the repo root: the one source of truth for the kernel version the driver is built against. Bumping it is the only way to pick up a new kernel, so a driver change and its kernel dependency always land together in one bisectable diff. Pinned to the kernel main SHA whose napi surface the committed binding (landed by the SEA stack) was generated from. - `.github/workflows/kernel-e2e.yml` — reads `KERNEL_REV`, checks the kernel out at that SHA via a GitHub App token, builds the napi binding (`npm run build:native` against the pinned checkout, cargo via the JFrog proxy), asserts the committed binding still matches the pin (drift-guard: `git diff --exit-code native/sea/index.*`), and runs the SEA e2e suite (`tests/e2e/sea/**`) against the dogfood warehouse. Synthetic-success on plain PRs; real run in the merge queue or via the `kernel-e2e` label; change-detection auto-passes when no SEA-relevant files moved. The test step invokes mocha directly (`npx mocha --config … "tests/e2e/sea/**"`) rather than `npm run e2e -- <glob>`: routing a glob through the npm-script's inner shell mangles `**` and silently resolves to zero files (a false pass). - `native/sea/README.md` — documents the pin and how to match it locally. Stacked on the SEA async/options PR: the committed b4d8822 binding (and the driver code + test fakes that consume its `submitStatement`/metadata surface) land there, so `KERNEL_REV` and the binding are consistent at every commit and the drift-guard passes. Requires one-time repo-admin setup (GitHub App allowlist for the kernel repo, the `kernel-e2e` label, warehouse secrets in azure-prod) — see the workflow header. Co-authored-by: Isaac Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
msrathore-db
force-pushed
the
msrathore/sea-kernel-rev-pin
branch
from
37228bd to
e8dff95
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #413 (which lands the
b4d8822kernel binding + the driver code/test fakes that consume itssubmitStatement/metadata surface). Retargets tomainonce #413 merges.Problem
The SEA napi binding is built from the kernel's private Rust source, not a published/versioned artifact, and the
.nodebinary is gitignored. So:native/sea/index.d.ts/index.jscorrespond to — the pairing isn't reproducible/bisectable.Fix
KERNEL_REV— one 40-char kernel commit SHA at the repo root: the single source of truth for the kernel the driver builds against. Bumping it is the only way to pick up a new kernel, so a driver change + its kernel dependency land together. Pinned to the kernel main SHA whose napi surface the committed binding (landed by [SEA-NodeJS] SEA async execute, TLS & connection/statement options #413) was generated from..github/workflows/kernel-e2e.yml— readsKERNEL_REV→ App-token checkout of the kernel at that SHA →npm run build:native(cargo via JFrog proxy) → drift-guard (git diff --exit-code native/sea/index.*— fail if the committed binding diverged from the pin) → SEA e2e suite against the dogfood warehouse. Synthetic-success on PRs; real run in the merge queue /kernel-e2elabel; change-detection auto-pass.native/sea/README.md— documents the pin + how to match it locally.Why stacked on #413
The committed binding and the driver code that consumes it must move together. #413 bumps the binding to the
b4d8822surface (withsubmitStatement/AsyncStatement/metadata) and updates the test fakes to match. PinningKERNEL_REVto that SHA on top of #413 keeps the pin, the binding, and the consuming code consistent at every commit — so the drift-guard passes andmain's unit tests stay green. (Pinning on amainbase would refresh the binding ahead of the code that uses it and break the foundation'sFakeNativeConnection.)Validated locally
Reproduced the workflow's build+test path against a clean kernel checkout at exactly
KERNEL_REV: pin resolves + passes the regex;build:native→ binding loads; SEA e2e green (execution,results,interval-duration); regenerated binding byte-identical to #413's. The local run also caught and fixed two first-draft bugs (the zero-file glob, and a committed-binding drift now covered by the drift-guard).Caveats
@databricks/sql-kernel-*binary packages — a separate release-engineering step.This pull request and its description were written by Isaac.