Repository navigation
Define perry-ext governance and migrate non-core npm shims away from bundled Rust rewrites #5716
Description
Problem
Perry currently uses many perry-ext-* crates as bundled well-known native bindings for npm package names. The mechanism is useful for true native boundaries, but it has started to blur the product direction:
Perry should compile real TypeScript/JavaScript packages whenever possible. It should not drift into reimplementing npm package-by-package in Rust.
This is related to, but different from, #5422. #5422 covers build weight and default/release build taxonomy. This issue covers the compatibility strategy behind that weight: which perry-ext-* crates should actually exist in-tree, and which should be removed, externalized, or replaced by compiling the real npm package.
Current facts from the repo:
cargo metadatasees 76 workspace members.- There are 37
perry-ext-*crates. default-memberscurrently includes many extension/staticlib crates.docs/src/packages/porting.mdalready says pure TS/JS packages should go throughperry.compilePackages, while.node/node-gyp/ prebuild packages need a native boundary.crates/perry/well_known_bindings.tomlcurrently maps many ordinary npm package names to in-tree Rust crates.
The concern: if every useful npm package becomes perry-ext-<name>, Perry will not become a Node/TypeScript ecosystem compiler; it will become a growing collection of Rust rewrites of npm packages.
Desired policy
Adopt this rule for bundled native bindings:
- Prefer compiling the real JS/TS package source.
- Prefer improving shared Node/Web runtime APIs over package-specific rewrites.
- Use
perry.nativeLibrary/perry-ext-*only for:- true native addon or system-library boundaries;
- runtime-adjacent Node/Web APIs that many packages share;
- very small strategic out-of-the-box shims, with an explicit maintenance/build-cost decision.
- Heavy or domain-specific integrations should be official external packages, not part of Perry core.
Not removal candidates in this pass
These crates are runtime-adjacent. They may still need build-policy cleanup or eventual stdlib consolidation, but they are not the main “reimplement npm package-by-package” problem:
| Crate | Current mapping | Reason to keep in/near core for now |
|---|---|---|
perry-ext-events |
events |
Node event semantics are foundational and used by many packages. If changed, the likely move is into stdlib/runtime, not external package deletion. |
perry-ext-fetch |
node-fetch, fetch |
Fetch/Headers/Request/Response are shared Web/Node runtime APIs. The package alias may be questionable, but the capability is core. |
perry-ext-http / perry-ext-http-server |
http, https, http2 |
Node HTTP server/client compatibility is core ecosystem infrastructure. |
perry-ext-net |
net |
TCP sockets are foundational for DB drivers, HTTP stacks, Redis, etc. |
perry-ext-streams |
streams |
Stream semantics are foundational for Node compatibility. |
perry-ext-ws |
ws |
WebSocket support is common runtime infrastructure; reassess later whether ws package compatibility should be source-compiled instead. |
perry-ext-zlib |
zlib |
Compression is a Node core API surface. |
Candidate removals / externalizations, case by case
“Remove” here does not mean abruptly break users. It means stop treating the crate as a permanent in-tree well-known Rust rewrite. Each candidate should get a migration path: compile the real package, move to stdlib/runtime, or publish an official external perry.nativeLibrary package.
A. Pure JS/TS packages that should be compiled from source
These are the strongest candidates to remove from the in-tree perry-ext-* strategy. Keeping them as Rust rewrites rewards the wrong compatibility path.
| Crate | Mapped package(s) | Why it should not be a bundled Rust ext | Suggested replacement |
|---|---|---|---|
perry-ext-dotenv |
dotenv |
dotenv is a small JS package. Rewriting it in Rust proves little about npm compatibility. |
Compile real dotenv; improve package resolution/fs/env handling if needed. |
perry-ext-nanoid |
nanoid |
Small JS utility. Rust rewrite bypasses package compatibility work. | Compile real nanoid; fix crypto/random or ESM issues uncovered. |
perry-ext-uuid |
uuid |
Common JS package. Perry should support it through package source plus Web/Node crypto. | Compile real uuid; improve crypto.randomUUID/random bytes support. |
perry-ext-slugify |
slugify |
Small string utility. No native boundary. | Compile real package; improve string/unicode behavior if needed. |
perry-ext-exponential-backoff |
exponential-backoff |
Tiny control-flow utility. No reason for a native binding. | Compile real package; improve timers/promises if gaps appear. |
perry-ext-lru-cache |
lru-cache |
Pure JS data structure. Native rewrite avoids exercising class/Map semantics. | Compile real package; fix JS class/Map/runtime gaps. |
perry-ext-commander |
commander |
CLI parser should be a package-compile target; Rust rewrite hides process.argv/CJS/ESM issues. |
Compile real commander; improve process/argv/package interop. |
perry-ext-cron |
cron, node-cron |
Scheduler packages are JS logic plus timers/date parsing. | Compile real packages; improve timers/date/runtime behavior. |
perry-ext-dayjs |
dayjs, date-fns |
Date libraries are canonical pure-JS ecosystem packages. | Compile real packages; improve Date/Intl/module compatibility. |
perry-ext-moment |
moment |
Same as dayjs/date-fns; keeping a Rust clone does not prove npm compatibility. | Compile real moment; fix Date/locale gaps as needed. |
perry-ext-decimal |
decimal.js, bignumber.js |
These packages are JS libraries with well-defined semantics. | Compile real packages; improve numeric/object semantics. |
perry-ext-cheerio |
cheerio |
Cheerio is a JS package stack; a Rust scraper wrapper is a different implementation and may diverge. |
Compile real Cheerio or document unsupported gaps; improve package/runtime compatibility. |
perry-ext-validator |
validator |
JS validation library. Rust rewrite avoids regex/string/package compatibility work. | Compile real validator; fix regex/string gaps. |
perry-ext-jsonwebtoken |
jsonwebtoken |
Mostly package logic over crypto primitives. The shared crypto APIs should be the investment target. | Compile real package once crypto/key APIs are sufficient. |
perry-ext-axios |
axios |
Axios is an HTTP client package. Perry should make it work through fetch/HTTP/URL/streams, not special-case it. |
Compile real Axios; improve Web/Node HTTP compatibility. |
perry-ext-ethers |
ethers |
Large JS ecosystem library. Rewriting a subset in Rust hides the real gaps in crypto, bytes, BigInt, package resolution, etc. | Compile real ethers or maintain as external package only if a native boundary is justified. |
perry-ext-ratelimit |
rate-limiter-flexible |
Package-level application logic. No core runtime/native reason to live in Perry. | Compile real package; fix Map/timers/Redis integration gaps separately. |
B. Domain-specific integrations that should be external packages
These may require native/system implementations, but that is an argument for perry.nativeLibrary packages outside the core repo, not for keeping every integration in Perry core.
| Crate | Mapped package(s) | Why it should leave core | Suggested replacement |
|---|---|---|---|
perry-ext-bcrypt |
bcrypt |
Password hashing is useful but not Node core. It is a package-level native/crypto integration. | Official external @perryts/bcrypt or compile a pure JS/WASM-compatible package where viable. |
perry-ext-argon2 |
argon2 |
Same as bcrypt: important but domain-specific and not core compiler/runtime. | Official external @perryts/argon2. |
perry-ext-better-sqlite3 |
better-sqlite3 |
Native addon style package. Correct boundary is a native library package, not Perry core. | External perry.nativeLibrary package, or point users to a supported DB package. |
perry-ext-ioredis |
ioredis, redis |
DB/client library. Should be covered by net/TLS runtime plus real JS package or official driver package. |
Prefer pure TS/JS driver via compilePackages; otherwise official external package. |
perry-ext-pg |
pg |
Database client should not be a bundled compiler-core integration. | Compile real pg once net/TLS/buffer are sufficient, or maintain @perryts/postgres externally. |
perry-ext-mysql2 |
mysql2, mysql2/promise |
Same database-driver concern; package compatibility should ride on shared net/TLS/buffer APIs. |
Compile real package or maintain external @perryts/mysql. |
perry-ext-mongodb |
mongodb |
Large database driver; not core runtime. | Compile real driver where possible or maintain external @perryts/mongodb. |
perry-ext-nodemailer |
nodemailer |
Email transport stack is application/domain infrastructure, not Perry core. | Compile real package over net/TLS/streams, or external official package. |
perry-ext-sharp |
sharp |
Native addon / image-processing boundary. Valid native binding idea, but too domain-specific for core. | External @perryts/sharp / image package with explicit native dependency policy. |
perry-ext-pdf |
@perryts/pdf |
Official product package, not Node compatibility core. Keeping it in core couples release cadence unnecessarily. | Move to official external @perryts/pdf using perry.nativeLibrary. |
perry-ext-ads |
perry/ads |
Mobile ads are a product/platform integration, not compiler/runtime core. | Move to official external package, likely platform-specific. |
C. Framework/package adapters to reassess
| Crate | Mapped package(s) | Concern | Suggested replacement |
|---|---|---|---|
perry-ext-fastify |
fastify |
Fastify is a JS framework. If Perry has solid HTTP, streams, URL, timers, and package interop, the real package should be the goal. | Compile real Fastify or support a tiny external adapter package, not a bundled Rust rewrite. |
Proposed migration strategy
Phase 1: Policy and documentation
- Document the native-binding policy in
docs/src/native-libraries/overview.md. - Update
docs/src/packages/porting.mdto state thatperry-ext-*is not the default npm compatibility path. - Add a table listing every current
perry-ext-*crate with: package mapping, category, migration target, and current status.
Phase 2: Build-surface cleanup
- Remove non-core
perry-ext-*crates from rootdefault-members. - Keep release workflows explicit: bundled extensions must be built by a release list/script, not implicitly through the default workspace.
- Align this with Make Perry builds lighter for development and explicit for release artifacts #5422's dev/release profile work.
Phase 3: Compatibility radars
- Add a “pure npm package” radar that must not use
perry-ext-*. - Expand
scripts/node_core_subset.pyfor shared Node APIs that packages depend on. - Add a native-addon detector for
.node,binding.gyp,prebuilds,node-gyp, orgypfileso packages are classified correctly.
Phase 4: One-by-one migrations
For each candidate above:
- Try compiling the real npm package via
compilePackages. - Record the first blocking Perry gap.
- If the blocker is a shared runtime/API issue, fix that instead of preserving a package-specific ext.
- If the package is truly native/domain-specific, move it to an official external package with
perry.nativeLibrary. - Remove the well-known binding only after a documented compatibility path exists.
Acceptance criteria
- Perry has a documented policy for when an in-tree
perry-ext-*is allowed. - Every existing
perry-ext-*has a recorded classification and migration decision. - Pure JS/TS packages are tracked as
compilePackagescompatibility work, not Rust rewrite work. - Heavy/domain-specific native integrations have an externalization plan.
-
default-membersno longer pulls non-core extension/staticlib crates into the default developer loop. - Release workflows still explicitly build and validate any extensions Perry intentionally ships.
Non-goals
- Do not break existing users abruptly.
- Do not remove runtime-adjacent Node/Web compatibility work.
- Do not claim full Node compatibility requires zero native bindings; native addon boundaries still need a story.
- Do not duplicate Make Perry builds lighter for development and explicit for release artifacts #5422's build-profile/staticlib work; this issue should supply the extension-governance rationale that complements it.
Activity