Skip to content

fix(warehouse): stop offering one database as two sources - #1300

Merged
Gilbert09 merged 1 commit into
mainfrom
posthog/warehouse-dedupe-hosted-postgres
Sep 25, 2026
Merged

Gilbert09 merged 1 commit into
mainfrom
posthog/warehouse-dedupe-hosted-postgres

Conversation

@Gilbert09

Copy link
Copy Markdown
Member

Problem

Detection returns one entry per matched registry kind, and a hosted Postgres
provider does not replace the generic Postgres footprint: a Neon or Supabase
project still names its connection with DATABASE_URL or a pg-family
driver, so both the hosted kind and Postgres come back.

Nothing downstream treats that as one database. The standalone warehouse
command's prompt lists every detected kind as a source to set up, and the
seeded data-source step is handed the same list and asked to reach an outcome
for each kind. A run that answers both ends with two warehouse sources over
one database, importing the same tables twice — the user pays ingestion for
the duplicate and gets two copies of every table.

Why: this came out of reading the seeded data-source step's telemetry,
where a run reported connecting the same database under two kinds.

Changes

  • SourceDetector.supersededBy names the kinds that reach the same data
    through a more specific connector.
  • detectWarehouseSources drops an entry when one of its supersessors also
    matched, so the more specific kind is the one offered.
  • Postgres declares Neon and Supabase. No other entry sets the field, and
    the signals themselves are untouched — a project with no hosted provider
    still gets Postgres exactly as before.

Test plan

  • detect.test.ts: a Neon project and a Supabase project each offer only the
    hosted kind, a plain Postgres project is unchanged, and a registry check
    holds supersededBy to kinds that exist. The previous test asserting the
    double detection is replaced by these.
  • Unit suite over every warehouse-touching file green (32 files, 979 tests);
    tsc --noEmit clean; Prettier and ESLint clean on the touched files.

Created with PostHog Desktop

🤖 Generated with Claude Code

A project on Neon or Supabase still names its connection with the generic
Postgres signals (`DATABASE_URL`, a `pg` driver), so detection returned both
the hosted kind and `Postgres`. Every consumer treats a detected kind as a
source to create, and the seeded step is asked to reach an outcome for each
one — so the same tables were imported twice.

`SourceDetector.supersededBy` names the kinds that reach the same data through
a more specific connector, and the detection loop drops an entry when one of
them also matched. Postgres declares Neon and Supabase; nothing else changes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Generated-By: PostHog Desktop
Task-Id: 03c0f29b-d484-443b-abe3-aa2c93052391
Copilot AI lite review requested due to automatic review settings September 22, 2026 11:55
@Gilbert09 Gilbert09 self-assigned this Sep 22, 2026
@Gilbert09
Gilbert09 requested a review from a team as a code owner September 22, 2026 11:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

@Gilbert09 Gilbert09 added the stamphog Auto-merge stamp for automated PRs label Sep 22, 2026 — with PostHog

@edwinyjlim edwinyjlim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

robot says there might be some edge cases where Supabase matches on any SUPABASE_* key or supabase-js that are only used for auth/storage, but the product DB is a separate Postgres (e.g. Prisma + pg on RDS)

@Gilbert09
Gilbert09 merged commit 6f562a1 into main Sep 25, 2026
21 checks passed
@Gilbert09
Gilbert09 deleted the posthog/warehouse-dedupe-hosted-postgres branch September 25, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stamphog Auto-merge stamp for automated PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants