Skip to content

[bug] setup.local.sh: migrations fail (no pg driver for drizzle-kit, and Postgres 17 vs migration 0122) #8252

Open

Description

@MagnusPladsen

Where did this happen?

Building from source

Version

main at 9a50076 (macOS, Apple Silicon, Bun 1.3.10, Docker Desktop)

What happened?

./.superset/setup.local.sh on a fresh clone finishes with Apply migrations and Seed dev account failed, so the dev app has no local account to sign in with. There are two independent causes.

  1. drizzle-kit uses the Neon websocket driver against plain Postgres. packages/db runs drizzle-kit migrate against DATABASE_URL_UNPOOLED (postgres://…@localhost:<LOCAL_PG_PORT>/main, direct Postgres). drizzle-kit chooses its driver from whatever packages it can resolve. Neither pg nor postgres is installed in the workspace, only @neondatabase/serverless, so it logs Using '@neondatabase/serverless' driver and fails with ECONNREFUSED on the first query (CREATE SCHEMA IF NOT EXISTS "drizzle"). The comment in setup.local.sh says "migrate uses direct pg", but nothing provides that driver.

  2. A migration needs Postgres 18, but local Docker runs 17. docker-compose.yml pins image: postgres:17. Migration 0122_webhook_events_swap_to_short_retention.sql renames webhook_events_swap_id_not_null → webhook_events_id_not_null. Postgres only creates named NOT NULL constraints from version 18, so on 17 the constraint doesn't exist and the migration fails. Neon presumably runs 18, which is why this only shows up locally.

Expected: setup.local.sh migrates and seeds a fresh local stack without manual steps.

Steps to reproduce

  1. Fresh clone, bun install (or just run the setup script).
  2. ./.superset/setup.local.sh
  3. The summary lists Failed steps: Apply migrations, Seed dev account. Seeding fails with relation "auth.users" does not exist because migrations never ran.

What made it pass for me (local workaround, not a proposed patch):

  • Make pg resolvable from packages/db. drizzle-kit then logs Using 'pg' driver.
  • Run the stack on postgres:18, with PGDATA=/var/lib/postgresql/data to keep the existing volume mount working. With both changes, bun run db:migrate and bun run db:seed-dev succeed.

Possible fixes: add pg as a devDependency of packages/db (or set an explicit driver in drizzle.config.ts), and bump the compose image to postgres:18.

Logs, screenshots, or recordings

$ drizzle-kit migrate
Using '@neondatabase/serverless' driver for database querying
 Warning  '@neondatabase/serverless' can only connect to remote Neon/Vercel Postgres/Supabase instances through a websocket
DrizzleQueryError: Failed query: CREATE SCHEMA IF NOT EXISTS "drizzle"
  [cause]: AggregateError [ECONNREFUSED]

After making pg available, on postgres:17:

Using 'pg' driver for database querying
DrizzleQueryError: Failed query:
ALTER TABLE "ingest"."webhook_events" RENAME CONSTRAINT "webhook_events_swap_id_not_null" TO "webhook_events_id_not_null";

Activity

  1. github-actions commented on Oct 7, 2026

    @github-actions
    Contributor

    Summary

    On a fresh clone, ./.superset/setup.local.sh cannot migrate the local Docker database, so the dev-account seed also fails with relation "auth.users" does not exist. The code confirms both causes in the report, and they are independent. First, drizzle-kit has no TCP Postgres driver to load, so it uses @neondatabase/serverless. That driver connects over a websocket and cannot reach plain Postgres on localhost. Second, migration 0122 renames NOT NULL constraints by name. Postgres creates named NOT NULL constraints only from version 18, but the local compose stack runs Postgres 17.

    Affected code

    • .superset/setup.local.sh:83 points DATABASE_URL_UNPOOLED at direct Postgres (localhost:$LOCAL_PG_PORT). The comment at .superset/setup.local.sh:115 says "migrate uses direct pg". local_migrate (:159-167) runs bun run db:migrate.
    • The root package.json:57 maps db:migrate to bun run --cwd packages/db migrate, which runs drizzle-kit migrate (packages/db/package.json:43).
    • packages/db/drizzle.config.ts:5-11 passes only dialect: "postgresql" and dbCredentials.url. It sets no driver, so drizzle-kit uses whichever driver package it can resolve.
    • packages/db/package.json:47-60 lists @neondatabase/serverless and no pg or postgres. No package.json in the repo depends on pg or postgres, and bun.lock has no pg@ or postgres@ entries. Neon's driver is the only one drizzle-kit can find.
    • docker-compose.yml:3 sets image: postgres:17.
    • packages/db/drizzle/0122_webhook_events_swap_to_short_retention.sql:74-79 has six RENAME CONSTRAINT "webhook_events_swap_*_not_null" statements. Before Postgres 18, NOT NULL is not stored as a named row in pg_constraint, so on 17 the first rename fails with "constraint does not exist".

    Root cause

    1. Driver: The setup script expects migrations to go over direct TCP, but nothing installs a TCP driver. Deploys don't hit this: deploy-preview.yml:90 and deploy-production.yml:53 run drizzle-kit migrate against real Neon, where the websocket driver works.
    2. Postgres version: Migration 0122 was written against Postgres 18 behaviour (the comment at :71-73 assumes Postgres creates named NOT NULL constraints), and Neon runs 18. The local stack runs 17. Every migration up to 0121 applies on 17, and the run stops at 0122.

    Reproduction

    1. Fresh clone, then bun install.
    2. Run ./.superset/setup.local.sh. drizzle-kit logs Using '@neondatabase/serverless' driver and fails with ECONNREFUSED on CREATE SCHEMA IF NOT EXISTS "drizzle".
    3. Make pg resolvable from packages/db and run bun run db:migrate again. Migrations apply up to 0122, then fail on the first RENAME CONSTRAINT.

    Or, without drizzle: on a postgres:17 container, run CREATE TABLE t_swap (id int NOT NULL); ALTER TABLE t_swap RENAME TO t; ALTER TABLE t RENAME CONSTRAINT t_swap_id_not_null TO t_id_not_null;. It errors on 17 and succeeds on 18.

    Fix direction

    • Driver: Add pg (and @types/pg) as a devDependency of packages/db. Before merging, check which driver drizzle-kit 0.31 picks when both pg and @neondatabase/serverless are installed. If it picks pg first, preview and production migrations also move to pg over TCP against Neon's unpooled URL. That should work, but it changes the deploy path, including how the SET LOCAL timeouts in 0122 behave. If the deploy path should stay as it is, keep the pg change local, for example with a separate local-only config or script.

    • Postgres version: There are two options.

      • Move docker-compose.yml to postgres:18 to match Neon. Two catches with the official 18 image:
        • It moved the default PGDATA to /var/lib/postgresql/18/docker and expects the volume at /var/lib/postgresql. Either set PGDATA=/var/lib/postgresql/data, as the reporter did, or change the mount.
        • Existing per-workspace volumes (superset-<workspace>_superset_db_data) hold 17 data files, which 18 will not start on. Those workspaces need docker compose -p <project> down -v and a fresh migrate and seed. The setup script or docs/ should mention this.
      • Make 0122 work on both versions by wrapping each rename in a DO block that checks pg_constraint first. This edits a committed migration under packages/db/drizzle/, so it needs explicit maintainer sign-off. drizzle-kit records applied migrations by timestamp, so environments that already ran 0122 would not re-run it.

      Bumping to 18 is the cleaner fix, because later migrations may also depend on Postgres 18 behaviour.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions