You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Field stores cannot tell us which query is slow: preload pg_stat_statements in the managed store and surface it by role #3899
When a field user reports "the web viewer / MCP tools are slow", nothing in the product can say which query is slow. The managed store:
loads only timescaledb in shared_preload_libraries,
has log_min_duration_statement = -1,
and has no pg_stat_statements.
So the store-log sweep (#3021) has no slow-statement lines to classify, and get_store_metrics has no per-statement view.
Diagnosing the current slowness reports (#3895, #3896, #3897, #3898) needed the store owner's credential and a temporary per-role log_min_duration_statement = 0 to attribute time per tool. A field operator cannot do that, and should not have to.
What is already there
The bundled runtime ships the module. On DARLING01 (PostgreSQL 18.4, nightly 458), pg_stat_statements.dll and its control file are present, and pg_available_extensions lists pg_stat_statements 1.12, not installed.
Proposal
Preload and create it in the managed store. Add pg_stat_statements to shared_preload_libraries in the managed conf, and CREATE EXTENSION IF NOT EXISTS pg_stat_statements in the darling database.
The preload change needs a PostgreSQL restart, so it should ride a path that already restarts the store (install/upgrade), with the extension created on the next start.
The product's statements are parameterized, so the normalized query text carries no collected data.
Surface it. An MCP/web read (for example get_store_query_stats): top statements by total and mean time, calls, rows and shared blocks, split by role. The role split is the useful axis: viewer is the web viewer, mcp is MCP tools, the owner is collection and maintenance, so "the web viewer is slow" becomes a ranked list in one call. A reset stamp from pg_stat_statements_info belongs beside it, so a reader knows the window.
Log slow reader statements by default. For example, ALTER ROLE viewer SET log_min_duration_statement = '5s' and the same for mcp, set during role provisioning, plus a slow_statement class in the store-log sweep so those lines are counted and retained with text. This is cheap, needs no restart, and catches the statements that are approaching the 15 s statement_timeout before they start failing.
Document the BYO equivalent for stores the product does not manage.
Why now
#3895 and #3896 are both "invisible at our scale" defects that were only found by measuring. A reporting user with this in place could send us the answer instead of a symptom.
Confirmed on all three production stores (nightly 458, 2026-09-22): shared_preload_libraries = timescaledb only; pg_stat_statements 1.12 listed in pg_available_extensions, not installed. It blocked a real production attribution today — per-statement calls / mean / max / stddev for the #3893 rollup and the per-server health read could not be taken, so the #3893/#3901/#3902 attributions had to come from pg_stat_activity sampling (lower bounds only). With this issue shipped those would have been one query.
Implemented in PR #3904, and verified live on DARLING01 (dev build, 2026-09-22):
The v13 conf block was appended with the merged list timescaledb,pg_stat_statements, plus track_utility = off. The service-owned restart brought the postmaster up with the preload live.
The extension was created on start, and both SECURITY DEFINER readers are in place.
viewer and mcp carry log_min_duration_statement = 5s and log_parameter_max_length = 0. The store-object convergence pass ran 14 steps with 0 failed.
Zero role-DDL statements recorded, even though provisioning ran its ALTER ROLE ... PASSWORD statements with the preload active.
Once it reaches the production stores, the #3893, #3901 and #3902 attributions become a single call (order_by: mean_time or max_time), with per-execution means and maxima instead of pg_stat_activity lower bounds.
Follow-up: #3915 fixes the post-merge review findings on #3904 (details there). Filed alongside it: #3910 (plaintext passwords in provisioning), #3913 (plain-PostgreSQL convergence cadence), #3914 (compose/BYO web and MCP connect as the owner).
Activity
erikdarlingdata commentedon Sep 22, 2026
Confirmed on all three production stores (nightly 458, 2026-09-22):
shared_preload_libraries = timescaledbonly;pg_stat_statements1.12 listed inpg_available_extensions, not installed. It blocked a real production attribution today — per-statementcalls / mean / max / stddevfor the #3893 rollup and the per-server health read could not be taken, so the #3893/#3901/#3902 attributions had to come frompg_stat_activitysampling (lower bounds only). With this issue shipped those would have been one query.CHANGELOG: the store names its own slow queries (#3899)
erikdarlingdata commentedon Sep 22, 2026
Implemented in PR #3904, and verified live on DARLING01 (dev build, 2026-09-22):
timescaledb,pg_stat_statements, plustrack_utility = off. The service-owned restart brought the postmaster up with the preload live.SECURITY DEFINERreaders are in place.viewerandmcpcarrylog_min_duration_statement = 5sandlog_parameter_max_length = 0. The store-object convergence pass ran 14 steps with 0 failed.ALTER ROLE ... PASSWORDstatements with the preload active.get_store_query_statsover MCP returns ranked, role-attributed rows.DatabaseSizeSqlfrom Analysis 'latest value' reads scan a server's entire retained history: half of every analyze_server call, and the database-size fact counts dropped databases (all three SKUs) #3896 already shows at a 679 ms mean on this 9-server store.Once it reaches the production stores, the #3893, #3901 and #3902 attributions become a single call (
order_by: mean_timeormax_time), with per-execution means and maxima instead ofpg_stat_activitylower bounds.The store records its own per-statement timings: pg_stat_statements p…
erikdarlingdata commentedon Sep 22, 2026
Closed by the watcher: delivered in PR #3904, merged to dev.
erikdarlingdata commentedon Sep 22, 2026
Follow-up: #3915 fixes the post-merge review findings on #3904 (details there). Filed alongside it: #3910 (plaintext passwords in provisioning), #3913 (plain-PostgreSQL convergence cadence), #3914 (compose/BYO web and MCP connect as the owner).
The store's statement statistics survive their review: ALTER SYSTEM a…
The store's statement text and store log keep no literal: a one-pass …
Store-object convergence re-runs on a plain-PostgreSQL store too: the…
Store-object convergence re-runs on a plain-PostgreSQL store too: the…
Role provisioning sends SCRAM-SHA-256 verifiers, never a password, an…