Repository navigation
Compile a standard Express app: full blocker inventory (codegen emits undefined @perry_closure_* globals → linked-but-crashing binary) #3527
Description
Summary
Tracking issue: what it takes to compile a standard Express app with Perry.
I took a textbook Express 5 app, installed real Express (express@5.2.1, 65 packages in the tree, all plain CommonJS/JS), and drove it through perry compile, removing one blocker at a time to surface the entire chain rather than stopping at the first error.
Headline finding (the good news): Express's whole dependency tree is essentially TypeScript-subset-compatible — all 65 packages route to native compilation; none require the (removed) V8 runtime. Modern JS like Express really does compile the same as TS here. With a handful of escape hatches + two tiny dependency shims, Perry emits an 11.6 MB native binary (COMPILE_EXIT=0).
The catch: that binary is a false positive. The compiler tolerates 49 per-module codegen failures and links the broken objects in anyway, so the binary Bus error: 10 on launch with zero output. The real central blocker is a codegen bug, not the JS/TS divide.
Repro
// app.ts
import express from 'express';
const app = express();
app.get('/', (req, res) => res.send('Hello World!'));
app.get('/json', (req, res) => res.json({ message: 'hello', value: 42 }));
app.listen(3000, () => console.log('listening on port 3000'));npm install express # express@5.2.1, 65 packages
perry compile app.ts -o app_binTested on main @ bef6c3530, macOS arm64, Apple clang 21.0.0.
The blocker chain (in the order you hit them)
1. Bare import of a JS package → V8-removed gate
With no config, express/index.js (CJS) routes to the removed JS runtime:
Error: JavaScript runtime (V8) support has been removed... pulled in a JS runtime via: express/index.js [express]
2. #497 trust gate
Adding perry.compilePackages: ["express"] then trips:
package 'express' is in compilePackages but not in perry.allow.compilePackages
Fix: also set perry.allow.compilePackages, or PERRY_ALLOW_PERRY_FEATURES=1.
3. Opt-in is not transitive — must enumerate all 65 packages
Even with both flags set for express, all 28+ transitive deps individually route to V8:
still routed to runtime JavaScript because they are not in perry.compilePackages: accepts, body-parser, content-disposition, ... vary
You must list every package in the tree (direct + transitive) in both compilePackages and allow.compilePackages.
4. 🐛 The "*" wildcard is broken for compile-routing
compilePackages: ["*"] + allow.compilePackages: ["*"] passes the trust gate (no #497 error) but the wildcard is not honored by native-compile routing — express is treated as un-opted-in and immediately routes to V8:
Compile package: *
JS module: express -> .../express/index.js
Error: JavaScript runtime (V8) support has been removed... express/index.js [express]
So "*" is strictly worse than naming "express". This is a standalone bug worth fixing first — it's the natural "compile everything" ergonomic and it silently does nothing for routing.
5. new Function(...) / Function(...) refused (#1677) — eval-family
With the full tree enumerated, the first hard language error:
depdindex.js:436—new Function(...)(builds deprecation wrappers from a code string to preserve arity).function-bindimplementation.js:85—Function(...). This one is deep and ubiquitous (pulled in byget-intrinsic, which nearly everything depends on).
Both are trivially replaceable with ordinary closures. PERRY_ALLOW_EVAL=1 bypasses but yields non-functional behavior.
6. Unimplemented stdlib surface (#463)
After shimming depd: buffer.hasOwnProperty is not implemented (and likely more behind it). PERRY_ALLOW_UNIMPLEMENTED=1 skips the whole category to surface deeper errors.
7. 🐛 CENTRAL BLOCKER — codegen emits references to undefined @perry_closure_* globals
With eval + unimplemented + features all bypassed and depd/function-bind shimmed, the frontend fully succeeds (Found 129 module(s): 129 native, 0 JavaScript), then codegen produces invalid LLVM IR for 49 modules. Every failure is the same shape:
clang -c failed (status=exit status: 1)
.../perry_llvm_*.ll:9153:44: error: use of undefined value '@perry_closure_node_modules_body_parser_lib_read_js__7'
9153 | call void @js_register_function_name(ptr @perry_closure_..._read_js__7, ptr @.str.1, i32 10)
js_register_function_name is handed a @perry_closure_<module>__N global that the module's IR never defines. 49 modules fail this; the build links the surviving objects anyway and reports success → the resulting binary Bus error: 10 immediately.
This is the bug to fix to make Express actually work. Two parts:
- (a) Codegen must emit (or not reference) the
@perry_closure_*symbol consumed byjs_register_function_name. Likely a named-function-expression / closure-naming path that registers a name for a closure global that wasn't materialized in this module. - (b) A per-module
clang -cfailure must fail the whole compile, not silently drop the object and link a broken binary. Right nowCOMPILE_EXIT=0with 49 codegen failures is actively misleading.
49 failing modules across 30 packages:
body-parser (6), express (lib/application, request, response, utils, view — 5), negotiator (4), qs (3), router (3), body-parser/lib/types (4), mime-types (2), math-intrinsics (2), has-symbols (2), plus call-bind-apply-helpers, call-bound, dunder-proto, ee-first, function-bind, get-intrinsic, get-proto, http-errors, iconv-lite, inherits, object-inspect, on-finished, path-to-regexp, proxy-addr, raw-body, send, serve-static, side-channel, side-channel-list, side-channel-map, side-channel-weakmap, statuses (1 each).
The breadth (core utility packages like get-intrinsic/has-symbols/side-channel) suggests this is one codegen bug hit by a common JS pattern, not 30 separate problems.
8. Non-fatal noise (won't block, but worth tracking)
Warning: Could not resolve import 'ejs'— Express's optional view-enginerequire; harmless for a JSON/text API.Could not resolve import 'supports-color'— debug's optional color dep.- Many
unknown identifier 'window'/'navigator'/'document'/'localStorage'/'global'/'HTMLElement'/'Atomics'/'Float16Array'warnings — universal-module browser branches that don't execute server-side. Assignment to undeclared variable '$gOPD', creating implicit local(gopd) — likely benign but worth confirming it doesn't change semantics.
What "yes, Express compiles" actually requires
In rough priority:
- Fix About README #7(a) — the
@perry_closure_*undefined-symbol codegen bug. This is the one that matters. Without it, nothing downstream runs. - Fix About README #7(b) — make per-module
clang -cfailure abort the build. The silent-link-broken-binary behavior masks About README #7(a) as success. - Fix useEffect + setState panics with RefCell already borrowed on macOS ARM64 #4 — honor
"*"in compile-routing (or reject it loudly so it's not a silent no-op). - Tracker: AOT-first eval/new Function strategy — evaporate runtime dynamism at build time #1677 — recognize/auto-rewrite the trivial
new Function/Functionarity wrappers indepdandfunction-bind(both are 1-liners replaceable with closures), or document them as known-good shims. - Compile-time error for unimplemented Node / Web APIs #463 — fill the stdlib surface gaps (
buffer.hasOwnPropertyand whatever's behind it once About README #7 is fixed and real modules execute). - Ergonomics — transitive opt-in: enumerating 65 packages by hand is impractical. Consider auto-including the transitive closure of a named package, once trusted.
- Then re-test runtime:
app.listen+ routing +res.send/res.json. Rawnode:httpis already well-supported, so the foundation is there — but it's untested for Express until About README #7 lands.
Notes / methodology
- Each layer was verified empirically by re-running the compile with the prior blocker removed (escape-hatch env vars + minimal closure-based shims for depd/function-bind). Logs available on request.
- The two shims (depd, function-bind) are semantically-equivalent ordinary-closure rewrites of eval-based arity tricks — exactly the kind of thing Tracker: AOT-first eval/new Function strategy — evaporate runtime dynamism at build time #1677/feat(codegen): const-fold literal new Function bodies to native functions #1679 could auto-handle.
- All work done in an isolated worktree; no repo changes.
Activity
proggeramlug commentedon May 31, 2026
Cross-references — the central codegen bug (#7) is a regression / incomplete fix
The
use of undefined value '@perry_closure_*'failure is not new — it's the same class as two already-closed bugs, both in the perry-codegen closure collector:use of undefined value '@perry_closure_*'on Effect #318 — "perry-codegen: closure-collector misses HIR variants —use of undefined value '@perry_closure_*'on Effect" (CLOSED)arr.map((p, i) => { … })emits LLVM IR referencing undefined @perry_closure_"* (CLOSED)The fix for those was to teach the closure collector about additional HIR variants. The Express dependency tree clearly exercises more HIR shapes the collector still misses (49 modules across 30 packages, many of them tiny utility files like
has-symbols/shams.js,math-intrinsics/sign.js,side-channel*). So #7(a) here should likely re-open / extend the #318/#343 work rather than be treated as a brand-new bug. The leftover.llfiles in the log paths give a direct minimal repro for whichever HIR variant each one is hitting.Other related existing issues (context, not duplicates)
js_node_http_create_servercodegen'd butperry-ext-http-servernot linked #846 (CLOSED) — "express link-fails:js_node_http_create_servercodegen'd butperry-ext-http-servernot linked" — earlier Express attempt, different (link) layer.perry.nativeLibraryandperry.compilePackages#497 (CLOSED) — the trust-allowlist gate (blocker linux compilation and README #2 above).new Functionfamily (blocker fetch linker error on macOS ARM64 with perry-react — _js_fetch_with_options not found #5: depd, function-bind).buffer.hasOwnProperty).Net: the one new, high-leverage bug worth opening fresh is #4 (the
"*"wildcard compile-routing no-op). Everything else maps onto existing issues; the make-or-break item is re-extending the #318/#343 closure collector to cover the HIR variants Express hits.proggeramlug commentedon May 31, 2026
Update: root-caused + fixed the central codegen bug (#7) — Express tree 49 → 1 codegen failures
Dug into blocker #7 (the
use of undefined value '@perry_closure_*'cascade). It is not a missing arm in the closure collector (that walker is exhaustive since #318/#343). It's a registration/materialization mismatch in display-name emission.Root cause
crates/perry-hir/src/lower/expr_function.rs:556-560— every named function expression unconditionally recordsclosure_display_names[fid] = "<name>", regardless of how it's used.crates/perry-codegen/src/codegen/artifacts.rsbranch (c) (~L1296-1312) iterates all ofhir.closure_display_namesand emits ajs_register_function_name(@perry_closure_<mod>__<fid>, …)for each — without checking the closure was actually materialized as an LLVM global.For
module.exports = function sign(){}, CJS export lowering materializes a different fid for the function; the original named-fn-expr fid is stale (never compiled viacompile_closure). Branch (c) registers a name for that stale fid → reference to a@perry_closure_*global that's never defined →clang -cfails.Verified in the emitted IR for the 1-file repro:
@perry_closure_m_js__4/__5/__6aredefined, but__3appears only in a danglingjs_register_function_name(@perry_closure_m_js__3, …)call.Minimal repro (no node_modules)
→
error: use of undefined value '@perry_closure_m_js__3'. Discriminator:const sign = function sign(){…}(bound to alet) compiles fine; only themodule.exports =/ unbound-assignment form triggers it.Fix
In
artifacts.rsbranch (c), guard registration on the materialized closure set — theclosuresslice already iterated bycompile_closureat artifacts.rs:153:Branches (a)/(b) were already safe (they use
__perry_wrap_*symbols /Stmt::Let-bound closures, which are always materialized). Net diff: +14 lines, codegen-only.Verification (rebuilt compiler)
-1. ✅express@5.2.1, 129 modules, all hatches + depd/function-bind shims):clang -c failed49 → 1;use of undefined value '@perry_closure_*'49 → 0. ✅const f = function named(x){…}, plain arrow, top-level fn, class method, andarr.map((p,i)=>…)(the Codegen: arr.map((p, i) => { … }) emits LLVM IR referencing undefined @perry_closure_* #343 case) all still compile + run correctly (2 4 2 9 1,3,5). ✅The one remaining codegen failure is a SEPARATE bug
qs/lib/stringify.jsnow fails with a different, narrower error (no closure undef — that count is 0):qs'sstringifyis a recursive named function expression that calls itself with 18 arguments (stringify(obj[...], key, ..., sideChannel)at stringify.js:183/327). The callee resolves through aLocalGet(indirect call), and codegen doesn't support an indirect call site with that many args. This is a distinct high-arity indirect-call codegen limit — should be its own issue; it's the next blocker after the closure fix.Still also true (unchanged)
clang -c(or lowering) failure does not fail the overall build —COMPILE_EXIT=0and a binary is written even with the qs failure → the linked-but-broken binaryBus errors at runtime. That masking should be fixed too."*"wildcard no-op), fetch linker error on macOS ARM64 with perry-react — _js_fetch_with_options not found #5 (eval in depd/function-bind), fetch().then() callbacks never fire in macOS native UI apps #6 (buffer.hasOwnProperty) unchanged.Change is in a worktree, uncommitted. Happy to open a PR for the closure fix (small, self-contained, regression-clean).
fix(codegen): guard closure display-name registration on materialized…
proggeramlug commentedon May 31, 2026
Update: the remaining qs codegen failure is fixed → PR #3539
The one codegen failure left after the closure-display-name fix (#3537) —
qs/lib/stringify.js→Call callee shape not supported (LocalGet) with 18 args— is now fixed in #3539.Cause: indirect/closure calls (callee via a local, e.g. a recursive named function expression self-call) were capped at 16 args in
try_lower_closure_call_fallthrough. Above 16 it bailed. Two latent runtime issues compounded it:js_closure_call_array's catch-all arm dropped args past index 15, anddispatch_with_arityonly had concrete arms up to 8.Fix (4 files): route >16-arg indirect calls through the variadic
js_closure_call_array(entry-block[N x double]alloca marshaling); makejs_closure_call_array's catch-all dispatch the full slice; extenddispatch_with_arity8 → 32 arms.Verified: 18- and 20-arg recursive self-calls compile and return correct numeric results (no dropped args); 3/16-arg unchanged. Clears the last codegen failure on the Express tree.
Status of the #3527 blocker chain
"*"wildcard compile-routing no-op — open, unfiled as its own issuenew Function/Functionin depd, function-bind) — open (Tracker: AOT-first eval/new Function strategy — evaporate runtime dynamism at build time #1677); trivial closure rewritesbuffer.hasOwnPropertyunimplemented (Compile-time error for unimplemented Node / Web APIs #463) — open@perry_closure_*— fixed, PR fix(codegen): guard closure display-name registration on materialized closures (#3527) #3537Next concrete blockers to runtime success: land #3537 + #3539, then re-test the Express tree end-to-end (will still need the depd/function-bind eval rewrites and may surface runtime-layer issues like #1021 async-lowering / #3272 IncomingMessage.headers).
proggeramlug commentedon May 31, 2026
proggeramlug commentedon May 31, 2026
✅ Verified: PR #3543 (correct number) is now confirmed. Real outputs from the rebuilt compiler: 3-arg→123, 16-arg→136, 18-arg→153, 20-arg→190 — all correct, no dropped args. The 18/20-arg recursive self-calls fail on origin/main and pass on the fix. This clears the last codegen failure on the qs module. (Earlier I cited #3539 by mistake and posted results before the build finished — both corrected.)
fix(codegen): support indirect/closure calls with more than 16 args (#…
fix(compile): fail build on per-module codegen failure (#3527) (#3546)
proggeramlug commentedon May 31, 2026
Blocker #4 (
"*"wildcard compile-routing no-op) fixed → PR #3551The
"*"/"@scope/*"wildcard inperry.compilePackagespassed the #497 trust gate but was a silent no-op for compile routing (every routing predicate —is_in_compile_package, the HIRCOMPILE_PACKAGES_OVERRIDElookup, and thecompile_packages.contains(name)sites — does exact package-name matching). SocompilePackages: ["*"]left every dep routed to the removed JS runtime.Fix: expand the wildcard into concrete installed package names at config-load time (walk the
node_modulestree incl. scoped + non-hoisted nested deps), before the #497 check and routing.Verified: Express 5.2.1 with
compilePackages: ["*"]+allow.compilePackages: ["*"]now routes 129 modules: 129 native, 0 JavaScript (identical to the hand-enumerated 65-package config) and compiles to a native binary with the documented eval/unimplemented hatches. Also removes the need to hand-enumerate all 65 transitive packages —"*"is now the one-line "compile everything" switch.#3527 chain status
@perry_closure_*— ✅ merged (fix(codegen): guard closure display-name registration on materialized closures (#3527) #3537)"*"wildcard compile-routing no-op — ✅ PR fix(compile): honor "*"/"@scope/*" wildcard in perry.compilePackages routing (#3527) #3551 (this)new Function/Functionin depd, function-bind) — open (Tracker: AOT-first eval/new Function strategy — evaporate runtime dynamism at build time #1677)buffer.hasOwnPropertyunimplemented — open (Compile-time error for unimplemented Node / Web APIs #463)With #3551, every self-contained codegen + compile-routing blocker in the chain is closed. The remaining items are large, separately-tracked language/stdlib features (#1677, #463) and runtime-layer work (#1021, #3272).
fix(compile): honor "*"/"@scope/*" wildcard in perry.compilePackages …
fix(runtime,codegen): make Object.hasOwn a real first-class value (#3527
fix(hir): resolve bare `eval` as a value instead of ReferenceError (#…
fix(hir): infer `bool && value` as the value's type, not Boolean (#3527…
fix(cjs): make module/exports reassignable so `var module = X` works (#…
fix(runtime): guard hasOwnProperty against misaligned non-object rece…
2 remaining items
fix(runtime): guard hasOwnProperty against misaligned non-object rece…
fix(regex): escape literal `[` inside character classes (#3527)
fix(runtime): guard hasOwnProperty against misaligned non-object rece…
proggeramlug commentedon Jun 5, 2026
Adding data points from a second real-world server stack — an HTTP API built
on Hono +
@hono/node-server+ Drizzle ORM +mysql2+stripe+zod(39runtime packages). Same exercise as this issue; some overlap, some new.
New, isolated bugs filed (minimal standalone repros, not specific to any app):
Array.prototype.pushwith >1 arg/spread fails codegen inside amethod/getter body (
array.push_spread expects exactly 1 arg, got 2). Top-levelworks. This was the only module-level codegen failure across all 507 modules
when every dep was wildcard-compiled — it comes from
zod'sParseInputLazyPath.get path().enumreferenced before its declaration → "unknownidentifier", reads lower to
0(silent miscompile).zod'sZodFirstPartyTypeKindtriggers ~90 of these, silently corrupting its type-kind dispatch.
enumreverse mapE[value]→undefined.globalunresolved #4511 — legacyescape()/unescape()→ReferenceError(used byqs); bareNode
globalunresolved.Overlap with the cluster here (#3828): the
function-bind/call-bind/get-intrinsic/object-inspectFunction(...)pattern reproduces via thestripe→qs→ es-abstract chain.mysql2adds another flavor:generate-functionbuilds row parsers with runtime
new Function(its hot path), which AOT can'tconst-fold (#1679).
Other mysql2-chain blockers seen:
safer-buffer(viaiconv-lite) callsprocess.binding('buffer')→ unimplemented(Compile-time error for unimplemented Node / Web APIs #463). It's a no-op shim over modern
Buffer, so a recognized package-replacementwould unblock the whole iconv-lite/mysql2 branch.
long(via mysql2) referencesWebAssembly, so the link step fails late withWebAssembly.* used but libperry_wasm_host.a not foundunless--enable-wasm-runtimeis passed — the error surfaces at link time rather than as an early compile hint.
Net: with bypass flags on, all application source + 506/507 dependency modules
compiled; the residual blockers are the four issues above plus this Function()/wasm
cluster.
fix(http): native IncomingMessage/ServerResponse/… expose a real `.pr…
fix(http): native IncomingMessage/ServerResponse/… expose a real `.pr…
perry-hir: fix ||/&& result-type inference dropping the left operand (#…
proggeramlug commentedon Jul 30, 2026
The original blocker chain is now resolved: #3537 fixed stale closure display-name registration, #3543 fixed high-arity indirect calls, #3546 made module codegen failures fatal, and #3551 fixed wildcard compile routing. The later #5739 work verified a real NestJS app on @nestjs/platform-express, including GET and POST routes, and that fixture now lives under tests/release/packages/nestjs-hello. Remaining broad eval and stdlib work is tracked separately, so this inventory no longer represents an open Express blocker. Closing as completed/superseded.
feat(compile): zero-config auto-compile + binding faithfulness marker
feat(compile): zero-config auto-compile + binding faithfulness marker
fix(compile): keep the zero-config default additive and surface versi…
fix(compile): keep the zero-config default additive and surface versi…