Skip to content

Replace wasm-pack #1836

Description

@max-sixty

Not the most urgent item, but would be good to keep our build steps as uncomplicated as possible, and remove dependencies that aren't maintained.

Currently we use wasm-pack to bundle prql-js. IIUC we're using it because

  • we previously wanted to create a simple npm package, which wasm-pack does well
  • then we decided to also create bundler & web targets
  • so we made our own npm package which contained all three
  • ...but Instead of using wasm-bindgen alone, we kept wasm-pack and added package.json scripts to move the files it output (link above)

For context, wasm-pack is a wrapper of wasm-bindgen and isn't really maintained, whereas wasm-bindgen remains well-supported.

As discussed in https://rustwasm.github.io/docs/wasm-bindgen/reference/deployment.html, I think it should be possible to remove this middle layer and use wasm-bindgen alone; likely using a similar set of steps to what we do now in package.json scripts.

This would be a good contribution for someone who's somewhat familiar with JS, and would like an early PR before diving into the PRQL compiler.

Activity

  1. max-sixty commented on Mar 23, 2023

    @max-sixty
    MemberAuthor

    There's now a library that will do build a wasm package as part of the build:

    Quoting from #2158

    https://crates.io/crates/substrate-wasm-builder

    It requires nightly, which we won't ever require, but also possibly it might be moving to not require nightly? paritytech/substrate#13580

    We're 2-3 months behind on the toolchain, so assuming this works, we could use this in a few months to replace wasm-pack. That would simplify the build a lot as well as solving this perf issue.

    So I'll mark this as postponed, but leave it open, and we can implement this when it's available. It'll make the builds faster, much much faster when there are no changes, and reduce our use of unsupported crates.

  2. changed the title [-]Replace `wasm-pack` with bare `wasm-bindgen`[/-] [+]Replace `wasm-pack`[/+] on Mar 23, 2023
  3. added a commit that references this issue on Jun 20, 2023
  4. max-sixty commented on Jun 22, 2023

    @max-sixty
    MemberAuthor

    Update:

    • substrate-wasm-builder is cool but not that practical for us — it doesn't generate the JS file. It compiles a .wasm artifact by creating a whole new crate at build-time, building it, and then copying the .wasm artifact back 1
    • https://github.com/rustminded/xtask-wasm looks good as a replacement but doesn't seem to handle generating the package.json etc, which wasm-pack does
    • More understanding in Building with `build.rs` wasm-bindgen/wasm-bindgen#3494 (reply in thread)
    • So — stepping back — except from the general overhead of having wasm-pack around, the main pain comes from running wasm-opt on each run — it has no caching, and so is run on every run.

    So I think for the moment, we can have a build option that runs with --dev, which skips wasm-opt, which we can run locally. And then we can return to this if the ecosystem's tooling improves.

    Footnotes

    1. Given that it doesn't create JS files, I'm not sure why it does this, rather than just compiling the main project with wasm — possibly there are parts of the project that want to use a default target but which want to use a .wasm binary. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions