Log-event hashes are keyed with a per-store secret the service keeps outside the store, so a store reader can no longer test guesses at the literals they hide (#4004)

[erikdarlingdata]

Closes #4004.

Why

collect.pg_log_events stored two unkeyed SHA-256 values over text a store reader can mostly rebuild:

• raw_line_hash, over the raw log entry with its literals;
• statement_fingerprint, over RedactStatement's output, which leaves $$ bodies and DO blocks as written.

Anyone who can read the row can enumerate candidate literals offline and test each one against the hash. #3996's review recovered a four-digit PIN from a failed UPDATE in 6 ms. Erik's ruling: use HMAC-SHA-256 with a per-store key that the service generates and keeps outside the database. New rows use it, and old rows keep their hash until retention drops them. No tool or viewer returns any hash.

What changes

• PgLogHashKey (shared collectors) computes both identities as HMAC-SHA-256, with a separate subkey for each, truncated to the same 32 hex characters as before, so columns and reads are unchanged.

  • The unkeyed PgLogTextRedactor.Fingerprint/RawLineHash and the keyless PgLogEventClassifier.Default are deleted, so the compiler rejects any path that would hash without the key.
  • The classifier stamps every event it claims. The writer that both transports share refuses an unstamped event.

• The key is loaded once, at worker start (DarlingLogHashKeyFile). The same instance goes to every run that hashes: pg_log_events on the pg_read_file route and on the RDS route.

  Distribution	Where	Protection
  Managed Windows	log-hash-key.dpapi beside pg-credential.dpapi	DPAPI LocalMachine; DarlingFileSecurity ACL (SYSTEM, Administrators, service account; no interactive read)
  Compose	log-hash-key in the darling-credentials volume	0600 at creation; The web dashboard and MCP server connect to a compose or bring-your-own store as least-privilege roles, not as its owner (#3914) #3983's directory check (re-set to 0700, judged before/after) and file check (regular file, no group/other bits)
  Bring-your-own	darling-keys/log-hash-key beside darling.json (.dpapi on Windows)	Directory created owner-only, so the same checks hold; the operator's config directory is left alone

• Missing and unreadable keys.

  • A key is generated only when nothing exists at the path. It is written owner-only from creation and moved into place without overwrite.
  • A key file that cannot be trusted, read, decrypted or parsed is refused, never replaced. The service logs the file, the reason and the remedy: fix access, or delete the file to rotate. pg_log_events then refuses on both routes with PgLogHashKey.UnavailableMessage, in BuildQuery, before it pulls 4 MB of log from the target.
  • A key found in a directory that other users could write to until this start is also refused. The web dashboard and MCP server connect to a compose or bring-your-own store as least-privilege roles, not as its owner (#3914) #3983 would regenerate there, but a key is never rotated silently.

• No surface returns a hash. statement_fingerprint is removed from get_pg_log_events, from the reader's page record and SQL (the SQL no longer selects it), and from the web log tab. raw_line_hash was already removed by Log messages show as PostgreSQL wrote them, the SQL in them stays normalized, and one statement's text never decides how another's is read (#3944) #3996. The lock-wait fact still groups on the fingerprint inside the store and emits only the count.

• Old rows keep their unkeyed hashes until the 30-day retention drops them. The first cycle after the upgrade stores each entry still in the tail's overlap window once more, and each statement shape's fingerprint changes once. This is the one-time discontinuity PostgreSQL deadlock reports store and show their SQL normalized, and no read returns a hash of the raw report (#4005) #4013 accepted.

• Hash census, nothing else to key.

  • plan_hash covers the stored, redacted plan.
  • deadlock_hash covers the stored, normalized graph since PostgreSQL deadlock reports store and show their SQL normalized, and no read returns a hash of the raw report (#4005) #4013.
  • text_hash is hashtext over the stored query_text.
  • PayloadDimensions.Digest covers the stored payload.
  • The pileup, incident and story hashes cover no raw text.

  None of these has a hidden input.

• Lite collects no PostgreSQL logs and hashes no raw log or statement text. Its only text hash is DuckDB hash(query_text) inside a COUNT(DISTINCT) over stored text, which is never stored or returned. So Lite gets no counterpart.

• Docs: the README covers the key in its credential-file table and in a new paragraph, and the compose file and Dockerfile comments name it.

• In-lane test fixes: AuroraOnlySqlIsGatedTests (Lite) and FleetIdentifierScrubTests (Darling) judged their .claude exclusion on the absolute path, so they scanned zero files when the repo was checked out under .claude/worktrees/. Both now judge the path below the repo root, and the repo's own .claude folder is still excluded. PgServerLogPathPinTests now gives its context a key.

Test plan

• PgLogHashKeyTests (new, 11 tests):

  • the HMAC differs from the unkeyed SHA-256 and is stable for the same key material (a restart);
  • two stores' keys give different hashes, on both columns and through the classifier;
  • the PIN attack: 10,000 guesses give 0 hits with the unkeyed hash and 0 with a wrong key, and the control hits with the right key;
  • the classifier stamps events, and the writer refuses an unstamped event;
  • BuildQuery refuses without a key;
  • the key is 32 bytes, and neither ToString prints key material;
  • a missing key is generated once, owner-only (Windows ACL via DarlingFileSecurity, or Unix mode 0600), and the same key comes back after a restart; on Windows the file is a DPAPI blob;
  • an unreadable key is refused and left byte-identical, a key ordinary users can read is refused and left byte-identical, and a directory at the path is refused and not overwritten;
  • the key sits beside each distribution's other secrets.

• Red-watch: with the raw-line hash reverted to unkeyed SHA-256, 3 of the 11 tests fail (the unkeyed-difference, two-stores and PIN-attack tests). Restored, all 11 pass.

• The existing no-hash pin in PgLogEventsPipelineTests is extended to statement_fingerprint in the tool, the reader's row type and the web tab.

• Targeted classes, including live ones on a PG 18.6 + TimescaleDB 2.30.1 rig: PgLogEventsPipelineTests, PgLogEventMetricsTests, PgTargetBlockingTests, PostgresEngineGateBehaviorTests, DarlingCollectorRunnerTests, ComposeStoreRolesLiveTests, DarlingStoreLoginsTests, DarlingManagedRolesTests, DarlingFileSecurityTests, ServerPageTabsTests, McpPayloadContractCensusTests, DocCommentHygiene.

• Full Darling.Tests with DARLING_TEST_PG: 13,025 tests, 16 skipped (environment-gated), 4 failed. None of the four is caused by this change:

  • FleetIdentifierScrubTests (2): the worktree-path exclusion, fixed in this PR.
  • RefreshCeilingStalenessTests.TheSpacingFinding_IsCalledPerLightRefresh_OnTheSameWatch: it expects CRLF in DarlingWorker.cs, and my working copy was briefly LF. The committed content is unaffected.
  • TrendPayloadBudgetLiveTests: a transient Exception while reading from stream from the rig on get_pg_io_trend.

  Re-run after the fixes, those classes plus the touched ones pass 215/215.

• Full Lite.Tests, since the change touches the shared collectors: 5,173 tests, 4 failed, all in AuroraOnlySqlIsGatedTests (the worktree-path exclusion, fixed in this PR). Re-run 9/9.

• Both builds report 0 Warning(s).

🤖 Generated with Claude Code

https://claude.ai/code/session_01GdmA4ND1wLSqA91ax1m4xv