Repository navigation
JSON.parse + iterate + field access silently drops matches at scale #44
Description
Repro
// Build input text manually (avoids JSON.stringify, tracked separately)
let parts: string[] = ['['];
for (let i = 0; i < 50000; i++) {
if (i > 0) parts.push(',');
parts.push('{"id":' + i + ',"name":"Record_' + i +
'","email":"r' + i + '@x.com","country":"US",' +
'"tags":["a","b","c"],"active":true,' +
'"addr":{"street":"' + i + ' Main","city":"Springfield","zip":' + (10000 + i) + '}}');
}
parts.push(']');
const text = parts.join('');
console.log('input bytes:', text.length);
const parsed = JSON.parse(text);
console.log('parsed length:', parsed.length);
let count = 0;
for (let i = 0; i < parsed.length; i++) {
if (parsed[i].active === true) count++;
}
console.log('expected', parsed.length, 'active, got', count);Expected
expected 50000 active, got 50000
Actual (Perry 0.5.30)
parsed length: 49936
expected 49936 active, got 56
Two distinct problems surface here:
parsed.lengthis 49936 instead of 50000 — parse itself drops ~0.1% of records silently.- Of the 49936 records
parsedid return, only 56 have.active === true— a loss of 99.9% on field-read.
Both counts vary slightly run-to-run, which is the tell: this is a GC-sweep timing issue, not a parser-state bug.
Observations
- Below ~2000 records with rich objects, works correctly (50k active = 50k matches).
- Simple
{id, active}objects work at 500k records. The problem scales with object complexity, not just record count — suggesting the sweep is freeing the parser's string/object allocs before the user iteration reaches them. - No panic, no stderr output — the corrupted
.activereads returnfalse/undefinedsilently.
Workaround (partial)
Access fields via explicit temporary variables or extract into fresh objects early. Below ~1000-record scale, building fresh object literals from parsed data (rather than mutating parsed records) is reliable.
Likely same underlying root cause as #42 (large Buffer param) and #43 (JSON.stringify panic) — parser-allocated objects aren't reachable from the GC's conservative stack scan when held only in a local array-of-pointers.
Activity
proggeramlug commentedon Apr 15, 2026
Fixed in v0.5.33.
Two compounding bugs:
1. trace_array length cap.
trace_arrayincrates/perry-runtime/src/gc.rsshort-circuited withif length > 65_536 { return; }. The repro'sparts: string[]builder pushes ~100k entries (one opening bracket + one comma + one record per iteration × 50k = 100001), crossing that cap. Every string reference inpartswas invisible to the mark phase and got swept on the first malloc-count-triggered GC. Raised the cap to 16M (still well below any plausible corrupted value).2. Arena-block-persistence hole. During tight
arr.push(new_obj)loops, the newly-allocated object lives only in a caller-saved register between allocation and the write intoarr. Conservative stack scanning (setjmp captures callee-saved regs only) doesn't see it as a root. The arena block containingnew_objstill survives because arena block reset is all-or-nothing and other objects in that block are reachable fromarr— butnew_obj's malloc-allocated string children get swept anyway, leaving dangling pointers. Subsequent reads (.active === true) return garbage, which explained the silent drops.Added a block-persistence pass
mark_block_persisting_arena_objectsthat runs after the primary trace: identifies arena blocks containing any reachable object, then marks every remaining arena object in those blocks and traces its malloc children. Iterates to a fixed point since newly-marked objects may extend liveness into previously-dead blocks.Verified — 50k records parse to length 50000 with all 50000 active matches, matching Node exactly.
fix: JSON.stringify/parse on large arrays (closes #43, #44, v0.5.33)