Repository navigation
GC fix in #34 doesn't fully cover wrapper-object allocation in row decode (silent process exit on second 1k×20 query) #35
Description
Summary
The malloc-trigger fix in #34 (v0.5.25) hugely improves the bigint-allocation case (8.45 GB → 36 MB on 2M parses, thanks!) but the original @perry/postgres codec-bench scenario from #34 still degrades, just one iteration later and with a different exit signature:
- Before Memory leak / OOM in codec decode path on large result sets (segfault on second 1k×20 query, first 10k×20) #34: segfault (exit 139) on the second 1000-row × 20-column query.
- After Memory leak / OOM in codec decode path on large result sets (segfault on second 1k×20 query, first 10k×20) #34: silent clean exit (code 0) mid-loop on the second / third query — no exception, no log line, the awaiting
conn.query(...)promise just never resolves and the event loop goes idle.
The smaller tiny (SELECT 1) and param-1row workloads now loop 50× cleanly; the big-result one still doesn't survive past 1–2 iterations, and the 10000-row variant still dies on the first attempt.
Repro
@perry/postgres bench/bench-this.ts workload medium-1k-x-20:
// 1000 rows × 20 mixed-type columns (int8, text, numeric, bool repeating).
const sql = `SELECT
(n*1)::int8 c0, ('row-'||n||'-c1')::text c1,
(n::numeric/3)::numeric(20,5) c2, (n%2=0)::bool c3,
/* ... 16 more in the same pattern ... */
FROM generate_series(1, 1000) n`;
for (let i = 0; i < 50; i++) {
console.log('# iter', i);
const r = await conn.query(sql);
if (r.rows.length !== 1000) throw new Error('bad row count');
}Output:
# iter 0
# iter 1
# iter 2 ← never logs "iter 3"; process exits 0 here
No exception is thrown. The promise from the iter-2 conn.query(...) never settles. The driver's connection state is still alive (the bench's catch handler doesn't fire), but the event loop has nothing else queued, so the runtime cleanly exits.
Versus #34
The original ticket was a segfault (exit 139) — clearly memory corruption / OOM. After v0.5.25 the symptom became a silent zero-exit. That's actually a regression in observability — at least exit 139 made the failure mode visible. Suggests the GC sweep is now triggering and freeing something that shouldn't have been freed: the connection's state Map, or the pending-query slot, or some node in the awaited promise chain.
What changed between fail boundaries
| iters survived | Perry version |
|---|---|
| 1 + crash | 0.5.24 (pre-#34 fix) |
| 2 + silent exit | 0.5.25 (post-#34 fix) |
So #34 bought one more iteration. The leak still scales with cells decoded. Each row produces 20 wrapper allocations (Decimal for 5 numeric columns, plus per-row Record<string,unknown> and unknown[] and 20 cell entries each). With 1000 rows that's ~25k allocations per call — eventually collides with whatever the new GC trigger threshold settles to (survivors + 10k per the #34 commit message).
A possible mechanism: a wrapper holds a transient root that doesn't survive a sweep. When the sweep runs in the middle of decoding row N, the prior rows 0..N-1 stored in the resulting array are reachable only through one of those holders, which the sweep frees → the result array's tail entries become dangling → the next deref tears the connection state apart silently.
Suggested investigation
- Re-run the bench with
PERRY_LOG_GC=1(or whatever flag dumps GC events) and check the sweep that fires mid-decode. - Check whether the
RawRowaccumulator (Buffer[][]) is GC-rooted while the per-row-decoded array is being built, or only afterbuildQueryResultreturns. - Look for any conservative-stack-scan corner that, like the gc_malloc-entry case from Memory leak / OOM in codec decode path on large result sets (segfault on second 1k×20 query, first 10k×20) #34, doesn't hold the freshly-allocated wrapper across an arena alloc boundary in row 999's decoder.
Environment
- Perry 0.5.25 (after Memory leak / OOM in codec decode path on large result sets (segfault on second 1k×20 query, first 10k×20) #34 fix at 4f8f013)
- macOS 26 / Apple Silicon, debug + release both reproduce.
- Bench:
@perry/postgresbench/bench-this.tsmedium-1k-x-20workload.
Activity
Benchmark suite: @perry/postgres on Node/Bun/Perry vs pg + postgres.js
docs: Performance section + bench results from local Postgres run
fix: GC root-scan ws/http/events/fastify listener closures (refs #35,…
fix: GC root-scan net.Socket event listener closures (closes PerryTS#35…
fix(stdlib): #187 commander runtime + codegen — actually invoke .acti…
feat(perry-ffi+perry-ext-events): refs #466 — GC root scanner surface…
feat(perry-ext-cron): refs #466 — port cron (v0.5.564)
fix(hir,compile): #35/#321 disambiguate same-named cross-module funct…
feat(intl): implement Intl.Locale (#5344)
feat(intl): implement Intl.Locale (#5344)
fix(compile): #5918 — import_function_prefixes collides on exported n…
fix(gc): readline/stdin listener closures are moving-GC roots
fix(gc): readline/stdin listener closures are moving-GC roots (#6858)