Skip to content

Apply the sensitive-statement filter to collected PostgreSQL statement text (#4348) - #4383

Merged
erikdarlingdata merged 14 commits into
devfrom
fix/4348-collected-statement-text-filter
Sep 26, 2026
Merged

erikdarlingdata merged 14 commits into
devfrom
fix/4348-collected-statement-text-filter

Conversation

@erikdarlingdata

@erikdarlingdata erikdarlingdata commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

Refs #4348.

Collected PostgreSQL statement text now goes through the same sensitive-statement filter the store already applies to its own statements:

  • PgSensitiveStatementFilter holds the one pattern definition. StoreStatementStats uses it, and so do the two collectors that store statement text from a monitored server (PgStatementText and PgBlockingCollector).
  • A matching statement's text is replaced with a fixed placeholder before it is stored. Its query id and statistics are kept.
  • A one-time background job rewrites matching text already stored, using the same batching, per-server isolation and completion-marker discipline as the settings scrub.

CHANGELOG entry

  • Collected PostgreSQL statement text applies the same sensitive-statement filter as the store's own statements (#4383) - matching text is withheld before storage, and a one-time job rewrites rows stored earlier.

PostgreSQL's ~* regex engine, not System.Text.RegularExpressions, judges
the shared pattern: it accepts the POSIX ARE syntax without matching it,
so the corpus runs against a scratch PostgreSQL connection instead.
The corpus (#4348) found ALTER ROLE app SET work_mem = '64MB' flagged as
sensitive. Its first alternative matched CREATE|ALTER ROLE|USER|GROUP|
SUBSCRIPTION|SERVER by name alone, meaning to catch every credential-
bearing form of that DDL, but every such form already ends in a literal
that the remaining alternatives catch, so the blanket alternative added
false positives and nothing else. Removed; verified against the corpus
plus the existing StoreStatementStatsLiveTests cases on a scratch
PostgreSQL 18 container.
…4348)

PgStatementTextScrubLiveTests seeds matching and neighbor rows in both
collect.pg_statement_text and a compressed collect.pg_blocking_edges
chunk for two servers, runs PgStatementTextScrub.RunAsync, and checks
matching rows became the placeholder, neighbors are untouched, the
marker equals ScrubVersion, and a second run changes 0 rows.

Also rephrases two comments in PgStatementTextScrub.cs to a neutral
current-state wording.
@erikdarlingdata
erikdarlingdata marked this pull request as ready for review September 26, 2026 04:48
@erikdarlingdata
erikdarlingdata merged commit 34a57f6 into dev Sep 26, 2026
15 of 16 checks passed
@erikdarlingdata
erikdarlingdata deleted the fix/4348-collected-statement-text-filter branch September 26, 2026 04:48
erikdarlingdata added a commit that referenced this pull request Sep 26, 2026
…33Z UTC (#4440)

Moves the CHANGELOG entries carried in merged pull-request descriptions into [Unreleased]. The cut is PRs merged at or before 2026-09-26T17:37:33Z; the next splice starts after it.

- 128 PRs are spliced: Fixed 87, Changed 20, Added 17 and Security 5. Each entry sits at the top of its section, newest PR first, and its link definition joins the trailing block.
- 22 PRs have no user-visible entry (None, test-only, or deferred to a parent).
- Three entries had no section in their description, and each was assigned from its diff: #4363, #4383 and #4380 go under Security.
- Link fixes: the #4198 references point at the issue, and #4360's [#4203] label points at issue 4203.
- #4208's entry is taken from its diff.
- Two security entries are worded to the current state: #4351's rotation note, and #4363's journal-read line.
- Only CHANGELOG.md changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant