Repository navigation
csvlog-only PostgreSQL targets get a permanent stderr breadcrumb file, defeating PgNoStderrLogFileException's NOT EXISTS check #4019
Description
What is wrong
#3997 fixed the shared log tail (PgServerLogTail.TailCteSql) to exclude .csv/.json siblings from the
newest CTE, and added PgNoStderrLogFileException for the case where logging_collector is on but no
stderr-format file survives that exclusion (NOT EXISTS (SELECT 1 FROM newest)).
Live testing against a real PostgreSQL 18.6 target found that this second case almost never actually fires,
because it assumes a csvlog-only target has ZERO non-csv/json files in its log directory. In practice it has
ONE: PostgreSQL's syslogger writes a one-time "breadcrumb" the moment the collector determines stderr is not
among the final configured destinations:
2026-09-23 11:29:20.948 UTC [58496] LOG: ending log output to stderr
2026-09-23 11:29:20.948 UTC [58496] HINT: Future log output will go to log destination "csvlog".
Measured on a target started with logging_collector = on and log_destination = 'csvlog' from its very
first start (never having had stderr as a destination at any point): this file exists, is 170 bytes, is named
with the ordinary .log suffix, and is never appended to again. pg_ls_logdir() therefore always returns at
least this one non-csv/json file, so newest is never actually empty and PgNoStderrLogFileException does
not fire. The three collectors instead read this file every cycle, match nothing in it (a two-line breadcrumb
matches none of the deadlock/plan/log-event patterns), and read as a quiet target — the exact #3030-shaped
failure #3997 exists to name.
By contrast, a target that has stderr in log_destination from the start never writes this breadcrumb (there
is no "ending" to announce), which is how #3997's own live test (PgLogTailStderrSiblingLiveTests) tells the
two states apart today: it is the working control, not a counter-example.
Why this is not a mechanical follow-up to #3997's own fix
Detecting "this file is a birth-only breadcrumb, not real content" is a real design question with two
competing failure modes, not a mirror of existing code:
- A size or line-count heuristic ("small newest file => no real stderr") risks a FALSE POSITIVE on a
genuinely low-traffic target with stderr legitimately configured: a quiet server's.logfile can
legitimately stay small for a long time, and reporting a precondition gap on a healthy target would be a
new, different wrong answer, not an improvement. - Matching the breadcrumb's own text ("ending log output to stderr") is exact and low-risk for a
same-locale target, but every log read this product performs already has a known blind spot here:
pg_plan_capture_readiness'smessage_localefacet exists precisely because PostgreSQL translates its
message catalogue under a non-Englishlc_messages, and this breadcrumb is an ordinaryelog(LOG, ...)
message subject to the same translation. A naive English-only match silently fails to recognise the
breadcrumb on a translated server, understating the gap instead of naming it.
Whether the fix is a text match (with its own locale caveat stated plainly, matching message_locale's own
pattern), a byte-size floor chosen and justified with real measurements across a range of legitimate quiet
targets, or something else, is Erik's call.
Suggested next step
Measure how small a genuinely quiet (but real) stderr file gets over a realistic window, and decide whether a
byte-size floor is safe or whether a breadcrumb text match (English-only, with the locale caveat named the
way message_locale names it) is the better first cut.
Where the current, narrower behavior lives
PgServerLogTail.NoStderrLogFileMarkerSql, PgNoStderrLogFileException (PerformanceMonitor.Collectors).
The NOT EXISTS check is correct for the literal zero-file case (which upstream PostgreSQL discussion
suggests may become more common: see the in-flight patch "Fixed creation of empty .log files during log
rotation" on pgsql-hackers) and is not being removed by this issue - it is a real, if narrower than originally
scoped, defense in depth.
Activity
A csvlog-only PostgreSQL target is refused by name instead of read as…
erikdarlingdata commentedon Sep 23, 2026
Closed by the watcher: delivered in PR #4040, merged to dev.
4160: the csvlog cluster sets log_timezone = 'UTC' and also serves DA…
CI: start csvlog/jsonlog/stderr+csvlog PG clusters so the log-format …