incorporate features from wasmpeg #49

Activity

1. Project516 commented on Sep 25, 2026

   

   Project516

   on Sep 25, 2026

   OwnerAuthor

   More actions

   claude-opus-5-5 responding on behalf of project516

   Cloned wasmpeg shallowly and compared it against our Dockerfile/build scripts, packages/ffmpeg, packages/util, src/bind, the fiber pthread shim on build/ffmpeg-9-st (#41), and Node support (#33). Full detail and source links are in a local report, wasmpeg-research.html, not committed.

   Features

   wasmpeg does not solve the problem #41 is stuck on, it avoids it. It never runs FFmpeg's fftools CLI: a small C file (src/pipeline.c) calls libavformat/libavcodec/libavfilter/libswscale directly, frame by frame, synchronously. No pthreads, no scheduler, no SharedArrayBuffer, no JSPI. Their only Asyncify use is for awaiting WebGPU device/queue calls, unrelated to the CLI's threaded scheduler. Their vendored FFmpeg is n9.0.1.

   Ranked, with detail and effort estimates in the report:

   1. The "skip the CLI, expose primitives" architecture itself. Worth a design conversation on whether build(st): move the st core to FFmpeg n9.0.2 on a fiber pthread shim #41's fiber shim is solving a harder problem than we need, versus adding a narrower decode/encode/probe API alongside the existing CLI-compatible one (can't replace it, AGENTS.md rules out breaking API changes).
   2. Splitting LGPL and GPL build outputs into separate build dirs/artifacts (PRESET=lgpl vs PRESET=gpl in their scripts/build.sh), so nothing GPL-encumbered ever lands in an LGPL-only build. We currently only ship one, always-GPL tier.
   3. Data-driven codec presets (docs/formats/codecs.md) driving real bundle-size differences: their combined CPU+WebGPU build is 3.2 MB gzipped, cpu-only about half; our single-thread ffmpeg-core.wasm gzips to roughly 10.2 MB in a local build. Most of the gap is codec/tool selection and dropping the CLI, not a compression trick.
   4. A typed, promise-based high-level API (wasmpeg.decode()/.scale()/.probe()/.encode()) layered over their low-level ABI, versus our closer-to-raw exec(['-i', ...]) plus manual FS calls in packages/ffmpeg. Worth an additive convenience layer, not a replacement.
   5. Published FATE-based coverage/correctness tracking (COMPAT.md, CORRECTNESS.md) as living docs. Good practice, not code we can copy directly.

   Node support in wasmpeg is simpler than ours only because their synchronous C-ABI design has no worker/message-passing layer to Node-ify in the first place; it isn't directly comparable to what #33 already solved for our worker-based architecture.

   Licensing

   Verified against primary sources (FFmpeg's legal page and vendored LICENSE.md, the GPLv2/GPLv3/AGPLv3/LGPLv2.1 texts, and each library's own header).

   Our state: packages/core and packages/core-mt correctly declare GPL-2.0-or-later (Dockerfile builds FFmpeg with --enable-gpl plus x264, x265, libvpx, and lame; no --enable-nonfree anywhere). x264 and x265 are both GPL-2.0-or-later (verified from their own header grants, not GPLv2-only), libvpx is BSD, lame is LGPL. Nothing here is undistributable, only copyleft.

   Real gaps found, all fixable and none rising to "actively lying about the license":

   • Root LICENSE and the README badge both say MIT, for a repo whose most-downloaded packages are GPL-2.0-or-later.
   • Neither packages/core nor packages/core-mt ships a LICENSE file in its npm tarball (npm pack --dry-run shows only dist in files); the only license text in the repo is the root MIT one, under a different license than what's actually being distributed.
   • No NOTICE file anywhere explaining the GPL basis, unlike wasmpeg's own NOTICE (see below).

   wasmpeg is confirmed LGPL-2.1-or-later, verified directly (not just from the issue): full LGPLv2.1 text at their root LICENSE, "license": "LGPL-2.1-or-later" in their package.json, and a NOTICE file naming every file they modified in their vendored FFmpeg fork plus the pinned commit as their source offer. Their @wasmpeg/core package explicitly ships LICENSE and NOTICE in its files array. Copying or adapting their code carries that LGPL-2.1-or-later forward onto whatever file it touches; copying the architectural idea without the code carries no obligation.

   On AGPL-3.0: mechanically we can relicense the MIT-covered wrapper code (MIT permits relicensing, provided the original copyright notice is preserved). Combining AGPLv3 with our GPL-2.0-or-later core is legally compatible, since GPLv2-or-later permits upgrading to GPLv3 terms, and GPLv3 §13 grants explicit interoperability with AGPLv3. No library in our current build is GPLv2-only, which is the one thing that would actually block this. That said, I don't recommend a blanket AGPL-3.0 relicense: it doesn't add protection beyond what an accurate GPL-2.0-or-later core license already gives, and its distinguishing network-copyleft clause has little effect on code that mostly runs client-side in a browser. The core and wrapper packages carrying different licenses (GPL-2.0-or-later core, MIT wrapper) is correct and matches wasmpeg's own pattern of a real legal line at the worker/message boundary.

   Recommendation: fix the labeling/packaging gaps now (per-package LICENSE files, a root NOTICE, correct the README badge), don't do a blanket AGPL move, and consider a separate LGPL-only build tier (drop x264/x265/libvpx-encode) as an additive option for users who can't take on GPL today. One thing that needs the owner's own judgment rather than a guess: whether "prefer AGPL" is really about network-copyleft specifically (a product/business call), and whether any docs/marketing site outside this repo states a project-wide license that would also need updating.

   Full detail, the ranked feature table, and all source links: wasmpeg-research.html in the repo root (untracked, not part of this PR/issue).
2. Project516 commented on Sep 25, 2026

   

   Project516

   on Sep 25, 2026

   OwnerAuthor

   More actions

   claude-opus-5-5 responding on behalf of project516

   Evaluated two more repos the same way as the wasmpeg report: openclaw/ffmpeg-wasm and wesbos/FFmpeg.wasm. Full detail, file paths, and source links in a report: https://share.project516.dev/f651304d

   Neither helps PR #41's actual blocker

   openclaw/ffmpeg-wasm ships FFmpeg n9.0.2 (the same release PR #41 is moving the st core to), but it does not attempt a scheduler-free single-thread CLI at all. Its Node and browser builds both use stock, unpatched FFmpeg source with vanilla Emscripten real pthreads (--enable-pthreads, -pthread -sPROXY_TO_PTHREAD=1, scripts/build.ts lines 66-78), which need SharedArrayBuffer. Their own docs confirm the browser build requires COOP/COEP headers (docs/domains.md), and there is no patch directory anywhere in the repo. Node works there without SAB friction only because Node's SharedArrayBuffer has no cross-origin-isolation gate to begin with, an environment difference, not a technique PR #41 can use in a plain browser tab. If anything this is a useful negative signal: another team building FFmpeg 9 to wasm in 2026 also landed on "just require SharedArrayBuffer," not a fiber/JSPI-style workaround.

   wesbos/FFmpeg.wasm is an explicit, credited fork of ffmpegwasm/ffmpeg.wasm (full history back to 2019, maintained by DreamOfIce after the original went quiet), last active 2023-07-25 at v0.13.0. Its core repo (DreamOfIce/ffmpeg.wasm-core) never got past FFmpeg 4.3.1 — its own README Todos list still has "Upgrade to FFmpeg@6" unchecked — so it never faced the fftools threaded-scheduler problem PR #41 exists to solve. Its st/mt split (wasm/build-scripts/build-ffmpeg.sh) just toggles -pthread at the emcc link stage with identical FFmpeg configure flags otherwise; without -pthread, pthread_create calls in FFmpeg's codec-level slice/frame threading degrade to running synchronously, which is safe pre-6.0 since fftools didn't yet spawn one thread per pipeline stage. That's the same shape our own st core already had on 5.1.x before PR #41, not new.

   Worth adopting anyway

   1. A CI step that scans the built dist/ for accidentally-included native executables (file on every output, reject ELF/Mach-O/PE32), from openclaw's .github/workflows/ci.yml. Cheap, portable to build-core/build-core-mt.
   2. NODERAWFS for direct filesystem access in openclaw's Node build (scripts/build.ts), letting the compiled tool read/write real host paths with no virtual-FS write/read dance. Relevant to issue feat(ffmpeg): typed helpers for probe, decode, and transcode over the exec API #50's typed helpers and our Node support (feat(ffmpeg): run the core in worker_threads under Node.js #33); Node-only, additive.
   3. A negative finding worth stating explicitly in our own NOTICE (PR chore: relicense to AGPL-3.0-or-later and credit upstream and third-party components #54): DreamOfIce/ffmpeg.wasm-core builds with --enable-nonfree for fdk-aac, which FFmpeg's own docs flag as not cleanly redistributable under GPL. We don't do this and shouldn't; worth a one-line note in NOTICE that AAC encode is intentionally left out rather than reached for via fdk-aac.

   Everything else in both repos (build-surface discipline, a lifecycle exit()/task-cancellation API, dual ESM/CJS import, custom core URLs) is either already covered by our own design (packages/ffmpeg already has AbortSignal support and terminate(), PR #33 already solves Node/ESM) or already tracked by issues #50-#52.
3. Project516

   closed this as completedin #54on Sep 25, 2026
4. Project516

   added a commit that references this issue on Sep 25, 2026

   test: run FFmpeg's FATE suite and benchmarks against both cores (#55)

   ...

   e947e90