Migrate database from SQL Server to PostgreSQL#5
Merged
Conversation
Replace Microsoft.Data.SqlClient with Npgsql across Dispatch.Data: - Connection factory, DatabaseInitializer (postgres maintenance DB, pg_database, CREATE TABLE IF NOT EXISTS, transactional DDL) and DatabaseHealth. - Rewrite all 7 schema migrations to PostgreSQL DDL (identity columns, boolean, timestamptz, "key" quoting, partial indexes; collapse the dynamic-SQL FK drop). - Port every repository query: MERGE→INSERT..ON CONFLICT, OUTPUT→RETURNING, TOP→LIMIT, DELETE TOP→ctid-subquery, ISNULL→COALESCE, SYSUTCDATETIME→now(), DATEADD→interval, IN @list→= ANY(@list), SUM(bigint)::bigint casts. - Storage/maintenance use pg_database_size / pg_total_relation_size. - Redesign size-pressure: opt-in configurable DB-size cap (default off) driven by pg_database_size, bounded overage purge + VACUUM FULL to reclaim. - Port the integration-test fixture to Npgsql; case-sensitive log search (accepted). All 312 tests pass against a real postgres:17 (32 DB integration tests included). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- docker-compose.yml: postgres:17 service with pg_isready healthcheck, Npgsql connection string; drop Azure SQL Edge. - Dockerfile / appsettings.json: reword SQL -> PostgreSQL/config table. - build.yml: postgres:17 service container, Npgsql DISPATCH_TEST_SQL, health-cmd. - All CI workflows (build/docker/appliance/installers/release/scripts) now run on self-hosted runners: ubuntu-latest -> [self-hosted, linux, x64], windows-latest -> [self-hosted, windows, x64]. Smoke-tested the full compose stack (Dispatch + postgres:17): schema init + all 7 migrations, /health connected via pg_database_size, live audit_log + relay_counters writes (ON CONFLICT upsert) through SMTP intake. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Windows installer: replace the SQL Express Burn bootstrap with a silent PostgreSQL install (installer/windows/postgres/InstallPostgres.ps1 + PostgresLauncher); WiX bundle chains it, MSI SQLCONN defaults to a loopback-trusted 'dispatch' role, functional-smoke size-pressure test reframed as the opt-in DB-size cap. - Linux installer (install.sh): --install-postgres/--db-password (apt/dnf postgresql, create dispatch role + DispatchLog db); dispatch-update.sh backup via pg_dump. - Appliance: bake in postgresql (build/provision/firstboot), per-VM role password, systemd postgresql; drop the Microsoft EULA/license-terms. - Docs: README, SPEC.md (schema DDL, bootstrap sections, size-pressure), RELEASING, installer README all moved to PostgreSQL/Npgsql. - CI workflow wording + docker.yml service (sql -> db); comment cleanups in WebEndpoints/PurgeWorker/IStorageReport/tests. Full suite (312 tests) green in Release against postgres:17. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The installers.yml full-install smoke assembled a tarball with only install.sh + dispatch.service, but install.sh installs dispatch-updater.service and dispatch-update.path too, so `install.sh --prebuilt` aborted at the unit step. Ship the same file set as the release tarball (units + updater script + public key) so the full-install smoke actually completes. Pre-existing gap, surfaced while running the Linux full-install smoke end-to-end. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces SQL Server / SQL Server Express with PostgreSQL (Npgsql) across the product, deployment, and docs. Fits the open-source relaunch: no proprietary DB dependency, no EULA, cross-platform.
What changed
Microsoft.Data.SqlClient→ Npgsql 10. Connection factory,DatabaseInitializer(postgres maintenance DB,pg_database, transactional DDL), and health probe rewritten.boolean,timestamptz,"key"quoting, partial indexes; dynamic-SQL FK-drop collapsed).MERGE→INSERT … ON CONFLICT,OUTPUT→RETURNING,TOP→LIMIT,DELETE TOP→ctidsubquery,ISNULL→COALESCE,IN @list→= ANY(@list),SUM(bigint)::bigintcasts,pg_database_size/pg_total_relation_size.VACUUM FULL.postgres:17), Dockerfile, CI DB service, Windows installer (silent PostgreSQL bundle replacing SQL Express), Linuxinstall.sh, and the virtual appliance.Validation
postgres:17(32 DB integration tests included)./healthreportsconnected:trueviapg_database_size, and SMTP intake writes counters + audit rows through theON CONFLICTupsert path.Notes for reviewers
ILIKEif desired.install.share written and syntax-checked but need a real Windows/Linux box.InstallPostgres.ps1has aREPLACE_WITH_PINNED_SHA256placeholder to fill at release time.relay_log(FK torelays) — identical on SQL Server.🤖 Generated with Claude Code