Option to generate code which works with other bundlers

[sgratzl]

so far the target for wasm-pack is hard coded to web:

rollup-plugin-rust/index.js

Line 148 in 6138501

"--target", "web",

However, I wanna build and rollup a library which then another one should should bundle. Thus, I need to set the target to bundler or nodejs.

It would be great if the plugin supports a simple option for that.

[Pauan]

Yes, that is correct, since --target web is the only target that works with Rollup.

Using target: "es" in Rollup works fine, and will produce ES6 modules which another bundler can bundle.

For NodeJS support you should set format: "cjs" in Rollup, and nodejs: true in this plugin.

[sgratzl]

ok thx for the response.

As far as I understand the --target web instantiates the WebAssembly by loading the file (using fetch for web or readFile for node). However, when I'm creating a library that is supposed to be consumed by other libraries, I don't know the final Webassembly name and path when I'm building the library. Do you have recommendations how to handle this case that the final bundler (who is using my library) copies the webassembly file at the right place with the right name, such that my library is going to find it?

My idea was to keep using --target bundler and mark the wasm file as external. Thus, rollup won't try to process copy or move this file but the final bundler would have to deal with that.

[Pauan]

Do you have recommendations how to handle this case that the final bundler (who is using my library) copies the webassembly file at the right place with the right name, such that my library is going to find it?

Unfortunately each bundler has their own way of loading static files, so there isn't a way to make it work with all of them. This is also a problem with other things like images and CSS.

So you will probably have to give instructions for each bundler on how to load the static file (e.g. Webpack would use asset modules).

My idea was to keep using --target bundler and mark the wasm file as external. Thus, rollup won't try to process copy or move this file but the final bundler would have to deal with that.

You can do that by manually running wasm-pack --target bundler and then copying the files into the build (by using something like rollup-plugin-copy-assets).

But I don't recommend that, since it will only work with Webpack (and only with experimental flags enabled).

Your best option is to use inlineWasm: true, this will inline the .wasm file into the .js file, so that way it doesn't need to be loaded separately. This will make the filesize larger, but it's the cleanest way to make it work in all bundlers.

[npbenjohnson]

I hacked something that makes it slightly more compatible with bundlers, but it's not polished enough to PR. Right now the wasm-pack output prefix is hardcoded to "index", changing it to "toml.package.name", and then having rollup emit a "filename" instead of a "name" for the wasm file makes it so that the filename in the generated wasm-pack library matches what rollup ends up creating.

// change
"--out-name", "index",
// to
"--out-name", toml.package.name,

// change
name: name + ".wasm"
// to
fileName: name + ".wasm"

// change anywhere there is "index" hardcoded in a string to toml.package.name

The plugin generated code also contains a default path and dynamic logic that second-level bundlers are unlikely to be able to update, so it needs to be removed (or probably disabled with config for backwards compatibility):

// change
                    export default async (opt = {}) => {
                        let {importHook, serverPath} = opt;
                        let path = ${import_wasm};
                        if (serverPath != null) {
                            path = serverPath + /[^\\/\\\\]*$/.exec(path)[0];
                        }
                        if (importHook != null) {
                            path = importHook(path);
                        }
                        await exports.default(path);
                        return exports;
                    };
// to (can probably be cleaned up further)
                    export default async () => {
                        await exports.default(path);
                        return exports;
                    };

That is enough to allow vite to rebundle without errors or modification. There is an additional hook for resolving import.meta.url which could possibly be used to customize the server path at runtime without breaking bundling by injecting some code the way importHook does, but I haven't figured out a pattern that works better than just using the above fixes:

// something like this,
resolveImportMeta(property, {moduleId}){
            if(property === "url" && state.tomlPackageNames[moduleId] !== null && options.metaUrlHook)
                return options.metaUrlHook(state.tomlPackageNames[moduleId]);
}

// moduleId is the import path for the wasm_pack js file, state.tomlPackageNames[moduleId] is toml.package.name
// whatever code is provided to the  metaUrlHook config would replace import.meta.url in the following code.
// I haven't come up with a way for this to be useful to the re-bundler yet, but it seems like it could be a good spot
// for environment config or globals
async function init(input) {
    if (typeof input === 'undefined') {
        input = new URL('whatever.wasm', import.meta.url);

[Pauan]

@npbenjohnson You might be able to get it working using the new experimental.directExports option.

[eugenesvk]

Am I correct that there is currently no set of flags/experimental options that would output a wasm module identical to wasm-pack build --target bundler (which is the default)?
And that it's blocked by rollup not implementing it rollup/rollup#2099

A little background for my slighly weird use case: I wanted to avoid JS and use Rust to create a user script (with inlined base64 wasm), this plugin nicely allowed to do that

But then some sites CSP-block wasm instantiation, so as a workaround I disable inlining and transpile .wasm file back to JS via yarn wasm2js

Then I modify the glue js files produced by wasm-pack build to source the new .wasm.js file, and then use rollup to bundle everything into a single file again by feeding as input the same wasm-pack glue files

But then this only works if I build the wasm itself with wasm-pack build --target bundler, which has imports like import * as $_user_script_wasm_bg_js from './user_script_wasm_bg.js';,

When I use this plugin, the script fails due to unknown wbg (the .wasm.js has imports like import * as wbg from 'wbg';)

Which as far as I understand is because of that hardcoded --target web argument (tried various combinations of directExports and rollup output format types, but those don't seem to work)

Or am I missing something simple?

[Pauan]

Am I correct that there is currently no set of flags/experimental options that would output a wasm module identical to wasm-pack build --target bundler (which is the default)?

The point of wasm-pack build --target bundler is to create code which will be bundled with Webpack.

Since this plugin is being used with Rollup, you're bundling with Rollup, so there's no need for the --target bundler output.

But then some sites CSP-block wasm instantiation, so as a workaround I disable inlining and transpile .wasm file back to JS via yarn wasm2js

I don't think there's much you can do about that. wasm2js won't work forever, because there are many features that are Wasm-only, and so they can't be compiled to JS.

That's really something that should be fixed in the user script system, allowing the user script to bypass the page's CSP.

I believe Chrome/Firefox extensions can bypass the page's CSP, so it should be possible.

What user script system are you using? TamperMonkey?

Or am I missing something simple?

I don't think you're missing anything, this Rollup plugin just isn't designed for your specific use case.

You could create a Rollup plugin that runs wasm-pack + wasm2js . Then you can just run the Rollup build like usual, instead of needing to manually modify the code.

[eugenesvk]

Thanks for your prompt and detailed response!

wasm2js won't work forever, because there are many features that are Wasm-only, and so they can't be compiled to JS.

Hopefully by then wasm will be a first-class citizen just like like JS so could sidestep the JS swap altogether!

That's really something that should be fixed in the user script system, allowing the user script to bypass the page's CSP.

I believe Chrome/Firefox extensions can bypass the page's CSP, so it should be possible.

What user script system are you using? TamperMonkey?

Violentmonkey

Afaik there is some Firefox bug that prevents it
violentmonkey/violentmonkey#1001
https://bugzilla.mozilla.org/show_bug.cgi?id=1267027

Though in general I don't understand why I can run JS (when CSP also only allows scripts from a specific website script-src site.com ), but not wasm. Also, I'm using Chromium, so will check with Violentmonkey's devs why wasm is different
(per conversation linked above TamperMonkey removes the CSP header altogether)
Although I've just checked the same code (with inlined wasm), and TamperMonkey has the same error Refused to compile or instantiate WebAssembly module because 'unsafe-eval' is not an allowed source of script in the following Content Security
So apprently it doesn't remove enough of those headers or something :)))

You could create a Rollup plugin that runs wasm-pack + wasm2js . Then you can just run the Rollup build like usual, instead of needing to manually modify the code

For now I'd like to try to make a simpler yarn sequential invokation of commands that would do those 2 steps and then call rollup to get the final file

[Pauan]

Though in general I don't understand why I can run JS (when CSP also only allows scripts from a specific website script-src site.com ), but not wasm.

It's because Wasm is a separate CSP directive:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src#unsafe_webassembly_execution

https://www.aaron-powell.com/posts/2019-11-27-using-webassembly-with-csp-headers/

It is very dumb. JS can do everything Wasm can do, so if it's possible to run JS scripts then it should be possible to run Wasm as well. But unfortunately the browsers decided to be ultra-strict for no reason.

So apprently it doesn't remove enough of those headers or something :)))

Perhaps they could automatically add in the wasm-unsafe-eval directive to the CSP.

[eugenesvk]

Though in general I don't understand why I can run JS (when CSP also only allows scripts from a specific website script-src site.com ), but not wasm.

It's because Wasm is a separate CSP directive:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/script-src#unsafe_webassembly_execution

https://www.aaron-powell.com/posts/2019-11-27-using-webassembly-with-csp-headers/

Thanks for the links, I've read them while trying to find out some workaround. What I still didn't fully get was - there is no unsafe-eval for JS, so by that logic JS shouldn't be able to run on the site either!

It is very dumb. JS can do everything Wasm can do, so if it's possible to run JS scripts then it should be possible to run Wasm as well. But unfortunately the browsers decided to be ultra-strict for no reason.

Indeed, the whole thing didn't make any sense to me. Then there is another "feature" I've read about re Manifest V3 where there is no differentiation between remote wasm code and embedded/hashed
from here

We were anticipating the introduction of another value or new directive that would allow us to restrict Wasm execution to sources loaded form the extension's origin, but unfortunately, that kind of control via CSP is still an unsolved problem. This created a situation where Manifest V2 extensions could use bundled Wasm but Manifest V3 extensions could not. Ultimately we decided it was best to allow extensions to set wasm-unsafe-eval and to adopt new CSP directives/values when they are introduced.

I think this lack of separation is he issue here, right? The browser doesn't treat same-origin wasm the same way as same-origin JS? That's why ↓ is needed

Perhaps they could automatically add in the wasm-unsafe-eval directive to the CSP.

Thanks for the tip, tried to do that with the CSP editing extension https://chrome.google.com/webstore/detail/modheader-modify-http-hea/idgpnmonknjnojddfkpgkljpfnnfcklj/related?hl=en-US, but it doesn't work properly - for some reason there is no "append" functionality for CSPs, so I can either override it to allow running inlined wasm (this works) or manually append, neither of which is a feasible option (interesting that even a dedicated extension doesn't work here)

[Pauan]

What I still didn't fully get was - there is no unsafe-eval for JS, so by that logic JS shouldn't be able to run on the site either!

unsafe-eval is for things like eval or new Function. Loading JS with <script> still works fine, even without unsafe-eval.

But you can't load Wasm with <script>, which is why Wasm needs wasm-unsafe-eval.

Then there is another "feature" I've read about re Manifest V3 where there is no differentiation between remote wasm code and embedded/hashed from here

Thankfully they changed their mind and decided to allow wasm-unsafe-eval for Chrome extensions.

I think this lack of separation is he issue here, right? The browser doesn't treat same-origin wasm the same way as same-origin JS? That's why ↓ is needed

No, it has to do with arbitrary code execution. By using eval or new Function it's possible for extensions to run any JavaScript code it wants.

But the Chrome extension team doesn't want that, because it causes a security risk. For example, the Chrome extension could use fetch to load some code from another website, and then use eval to run that code.

So the Chrome team wants to disable eval / new Function, which means you can only run JS code by using <script>, which is safer.

However, you can't run Wasm code with <script>, so that causes a problem. So that's why they decided to allow wasm-unsafe-eval, so that way it's possible to run Wasm code in a Chrome extension.

That's a different issue than what you are having. That issue is about the CSP for the Chrome extension, but your issue is about the CSP of the website. So your issue will need to be fixed separately.

[shniubobo]

Yes, that is correct, since --target web is the only target that works with Rollup.

The situation has changed quite a lot recently, and --target module could be possible as well:

• Rollup: added support for source phase imports in feat: external only Source Phase imports support rollup/rollup#6279
• Vite:
  • <8: uses esbuild and rollup, so also supports import source
  • >=8: uses rolldown, which in turn uses oxc, which has long since supported import source: feat(parser)!: stage 3 import source and import defer oxc-project/oxc#7706

--target bundler is not yet possible with either, but will soon be possible with vite:

• Rollup: same as before, esm integration still not supported. Related issues:
  • Better WebAssembly integration rollup/rollup#2099
  • Wasm plugin to support latest ESM Integration & Source Phase Imports rollup/plugins#1739
• Vite: a plugin for esm integration already exists, and a built-in implementation is awaiting approval: feat(wasm): direct .wasm imports (WASM ESM Integration) vitejs/vite#21779