Add wasm32-unknown-unknown compilation target support for matrix_sdk_sqlite by NoManColonies · Pull Request #6329 · matrix-org/matrix-rust-sdk

NoManColonies · 2026-03-20T17:23:43Z

Before starting, I apologize for any inconvenience caused by my non-native English.

This PR adds support for compiling matrix_sdk_sqlite to the wasm32-unknown-unknown target, enabling use in browser and similar WASM environments.

Currently, matrix_sdk in WASM is limited to:

in-memory state store/event cache, or
IndexedDB-backed state store with in-memory event cache

This means persistent event/media caching is not available without a custom implementation.
This PR introduces SQLite-backed persistence for WASM as an additional store option.

Summary of changes

Bump rusqlite to 0.38 (adds wasm32-unknown-unknown support via sqlite-wasm-rs)
- cache feature now required for prepared/cached statements
- fallible_uint feature required for unsigned integer casting
Add wasm32-wasi-vfs feature for WASM targets to enable a browser-compatible VFS
- Default backend: OPFS
- Other VFS implementation is split into a separate PR
Add web-time dependency for media retention policy serialization
Gate Tokio fs feature and deadpool-sync behind non-WASM targets
Replace Send with SendOutsideWasm where required for non-Send WASM environments
Use #[cfg_attr(...)] instead of unconditional #[async_trait] for target-specific behavior
Replace SyncWrapper<T> with RefCell<T> in the WASM deadpool adapter
- SyncWrapper<T> requires Send bound
- RefCell<T> provides equivalent interior mutability without Send bound

Notes for reviewers

Default VFS backend is OPFS; other VFS implementation is split into a separate PR
No public API changes beyond documented feature additions
- js feature is needed in order to compile to WASM target
Changes are largely conditional on target_family = "wasm"

I've documented the public API Changes in the appropriate CHANGELOG.md files
This PR was made with the help of AI
- This PR description was machine translated/assisted

Signed-off-by: Geng Kongpop geng@finnomena.com

codecov · 2026-03-20T18:15:49Z

Codecov Report

❌ Patch coverage is 82.35294% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.87%. Comparing base (85b8c83) to head (4bd5f07).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
crates/matrix-sdk-sqlite/src/crypto_store.rs	0.00%	0 Missing and 1 partial ⚠️
crates/matrix-sdk-sqlite/src/state_store.rs	75.00%	0 Missing and 1 partial ⚠️
crates/matrix-sdk-sqlite/src/utils.rs	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6329      +/-   ##
==========================================
- Coverage   89.88%   89.87%   -0.02%     
==========================================
  Files         376      376              
  Lines      103093   103097       +4     
  Branches   103093   103097       +4     
==========================================
- Hits        92670    92658      -12     
- Misses       6863     6870       +7     
- Partials     3560     3569       +9

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

codspeed-hq · 2026-03-20T18:28:52Z

Merging this PR will not alter performance

✅ 50 untouched benchmarks

_{Comparing NoManColonies:main (4bd5f07) with main (85b8c83)}

richvdh

Thanks very much for submitting this. We're quite excited by it, as it opens the door to things like being able to use the wasm bindings for matrix-sdk-crypto in node.js (matrix-org/matrix-sdk-crypto-wasm#168), and maybe even dropping support for indexeddb altogether.

Some questions, other than those below:

How were you able to test/build this? I tried things like cargo check -p matrix-sdk-sqlite --target wasm32-unknown-unknown and wasm-pack test --chrome --headless crates/matrix-sdk-sqlite, and got build errors.

Would you be able to add support to cargo xtask ci for checking and testing matrix-sdk-sqlite with the wasm target? I think you'll need to add a new entry to WasmFeatureSet.
Would you be able to break the changes down into smaller commits, to make them easier to review?

crates/matrix-sdk-sqlite/src/connection.rs

crates/matrix-sdk-sqlite/src/connection/default.rs

crates/matrix-sdk-sqlite/src/crypto_store.rs

richvdh · 2026-03-26T07:52:37Z

@NoManColonies I assume you're still working on this. Once it's ready for review, please rebase your changes on main and ping me.

NoManColonies · 2026-03-28T17:17:48Z

@richvdh

Thanks very much for submitting this. We're quite excited by it, as it opens the door to things like being able to use the wasm bindings for matrix-sdk-crypto in node.js (matrix-org/matrix-sdk-crypto-wasm#168), and maybe even dropping support for indexeddb altogether.

Some questions, other than those below:

How were you able to test/build this? I tried things like cargo check -p matrix-sdk-sqlite --target wasm32-unknown-unknown and wasm-pack test --chrome --headless crates/matrix-sdk-sqlite, and got build errors.
Would you be able to add support to cargo xtask ci for checking and testing matrix-sdk-sqlite with the wasm target? I think you'll need to add a new entry to WasmFeatureSet.

Would you be able to break the changes down into smaller commits, to make them easier to review?

Thank you very much for taking your time to review this PR and apologize for the late reply.
To address your questions:

How were you able to test/build this? I tried things like cargo check -p matrix-sdk-sqlite --target wasm32-unknown-unknown and wasm-pack test --chrome --headless crates/matrix-sdk-sqlite, and got build errors.

I initially tested the prototype in a temporary downstream project and I seem to have forgotten to properly configure the feature requirements in the upstream crate, my apologies. You should be able to build and test with the following commands:

Cargo check: cargo check -p matrix-sdk-sqlite --target wasm32-unknown-unknown --features js
Wasm-pack test: wasm-pack test --chrome --headless crates/matrix-sdk-sqlite --features js --release
- Currently, the OPFS backend has a harmless debug assertion that triggers at the end of test. So you may need to run the test under release profile

Would you be able to add support to cargo xtask ci for checking and testing matrix-sdk-sqlite with the wasm target? I think you'll need to add a new entry to WasmFeatureSet.

I’ve added tests as requested, but current VFS backend (OPFS) require dedicated worker context and is therefore not supported in a Node.js environment without polyfill. I have made a separate PR adding alternative backends that should be usable in Node.js.

Would you be able to break the changes down into smaller commits, to make them easier to review?

Apologies for the large initial commit, it was initially composed of many small commit that didn’t compile. I saw a note in CONTRIBUTING.md that requires all commits to compile successfully, and all initial changes were required in order to compile under WASM targets. If you'd prefer, I can break the initial commit down into a smaller one.

Since your previous comments, I have made the following changes:

I have refactored associated type of Manager trait in src/connection.rs to use a wrapper type around RefCell<T> and keeping the interface similar to SyncWrapper<T> in order to avoid most of conditional compilation. If you prefer a WASM connection manager entirely in a separate module, please let me know. I initially did not go this route, as it requires a lot of duplicate code in the connection manager that is largely the same as non-WASM targets WASM specific connection manager implementation has also been moved to a separate module.
I have created a utility function setup_db_fs in src/utils.rs for setting up filesystem under different compilation targets in order to avoid duplicate code in each module.
I have added tests for cargo xtask ci to cover the matrix-sdk-sqlite crate when compiled for wasm targets.
- The provided test vector isn’t compatible with SQLite compiled for wasm, so I dumped the database contents and converted them using online tool. The wasm tests now use this test vector instead of the previously provided one.
I have refactored VFS registration to avoid global installation which is harder to test and could limit downstream usage of SQLite.

If you have additional requests, please let know.

chore: address breaking changes from rusqlite feat: support alternative v/fs for wasm environment feat: support non-send future for wasm environment fix: solve incorrect error transformation feat: WIP support connection pool in wasm environment fix: use full-path method call for wasm environment + other fixes for wasm fix: more wasm related fixes feat: use RefCell to simulate interior-mutability in wasm environment fix: use implicit borrow instead of explicit call to `as_ref()` fix: more wasm related fixes fix: resolve ambiguity between lock guard and ref cell implicit reference + more wasm fixes fix: resolve wasm related lifetime issues chore: turn on serde feature for web time when compiled as wasm32 target chore: ran cargo check to verify Cargo.lock file chore: fix comment formatting

…g conditional compilation

richvdh

Thanks very much for the updates. I've taken a more detailed look. Generally it looks great, but I have some comments.

richvdh · 2026-03-30T20:41:03Z

crates/matrix-sdk-sqlite/src/connection/wasm.rs

+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! An implementation of `deadpool` for `rusqlite` for usage


It doesn't really implement the whole of deadpool, it's just a manager.

Suggested change

//! An implementation of `deadpool` for `rusqlite` for usage

//! An implementation of `deadpool::managed::Manager` for `rusqlite` for usage

richvdh · 2026-03-30T20:41:18Z

crates/matrix-sdk-sqlite/src/connection/wasm.rs

+// limitations under the License.
+
+//! An implementation of `deadpool` for `rusqlite` for usage
+//! in WASM environment.


Suggested change

//! in WASM environment.

//! in WASM environments.

richvdh · 2026-03-30T20:48:39Z

crates/matrix-sdk-sqlite/src/connection/wasm.rs

+//! Similar to the one implemented in `crate::connection::default`,
+//! we do not implement connection recycling here. Mostly due to
+//! [`managed::Manager`] trait expecting a future output with `Send`
+//! bound which is not available in WASM environment.


I don't really understand this comment. What does "do not implement connection recycling" mean? In what sense does Manager expect a future with Send?

richvdh · 2026-03-30T20:49:12Z

crates/matrix-sdk-sqlite/src/connection/wasm.rs

+/// The default runtime used by `matrix-sdk-sqlite` for `deadpool`.
+pub const RUNTIME: Runtime = Runtime::Tokio1;
+
+deadpool::managed_reexports!(
+    "matrix-sdk-sqlite",
+    Manager,
+    managed::Object<Manager>,
+    rusqlite::Error,
+    Infallible
+);
+
+/// Type representing a connection to SQLite from the [`Pool`].
+pub type Connection = Object;


Since this is the same in both implementations, maybe it could live in mod.rs?`

richvdh · 2026-03-30T20:54:06Z

crates/matrix-sdk-sqlite/src/utils.rs

+        }
+        .await
+    }
+}


I think this should probably live in connection::wasm

(and it probably needs a clearer name. I think probably just ConnectionWrapper would be fine, given it will be inside the connection::wasm module)

richvdh · 2026-03-30T23:11:26Z

xtask/src/ci.rs

+    if let Some(WasmFeatureSet::Sqlite) = cmd {
+        // Current VFS backend is not supported in Node.JS just yet.
+        // We force them to run in browser only for now.
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteAllFeatures), WasmTestRunner::Chrome)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteAllFeatures), WasmTestRunner::Firefox)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteCache), WasmTestRunner::Chrome)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteCache), WasmTestRunner::Firefox)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteState), WasmTestRunner::Chrome)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteState), WasmTestRunner::Firefox)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteCrypto), WasmTestRunner::Chrome)?;
+        run_wasm_pack_tests(Some(WasmFeatureSet::SqliteCrypto), WasmTestRunner::Firefox)?;
+        return Ok(());
+    }


can you put this next to the indexeddb equivalent?

richvdh · 2026-03-30T23:12:28Z

.github/workflows/ci.yml

+          # [m]-sqlite is not supported on Node.js yet, due to
+          # dedicated worker requirement for OPFS


I don't think this matters here.

Suggested change

# [m]-sqlite is not supported on Node.js yet, due to

# dedicated worker requirement for OPFS

richvdh · 2026-03-30T23:48:39Z

xtask/src/ci.rs

+        // SQLite WASM test suites has to be ran in release mode due to
+        // a harmless debug assertion when closing database.
+        //
+        // Ref: https://github.com/Spxg/sqlite-wasm-rs/blob/master/crates/sqlite-wasm-vfs/src/sahpool.rs#L672


why does this happen? The assertion is "close without open", but doesn't the database file get opened?

richvdh · 2026-03-31T00:00:18Z

.github/workflows/ci.yml

+      # Required in order to compile SQLite to WebAssembly
+      - name: Install emscripten toolchains
+        run: |
+          git clone https://github.com/emscripten-core/emsdk.git
+          cd emsdk
+          ./emsdk install latest
+          ./emsdk activate latest


This doesn't seem to be necessary when I run this locally. Are you sure it's needed?

crates/matrix-sdk-sqlite/src/connection/default.rs

NoManColonies changed the title ~~Pull changes from upstream~~ Added wasm32-unknown-unknown compilation support target for matrix_sdk_sqlite Mar 20, 2026

NoManColonies changed the title ~~Added wasm32-unknown-unknown compilation support target for matrix_sdk_sqlite~~ Added wasm32-unknown-unknown compilation target support for matrix_sdk_sqlite Mar 20, 2026

NoManColonies force-pushed the main branch from ad25eab to 44967f3 Compare March 23, 2026 04:09

NoManColonies marked this pull request as ready for review March 23, 2026 10:06

NoManColonies requested a review from a team as a code owner March 23, 2026 10:06

NoManColonies requested review from poljar and removed request for a team March 23, 2026 10:06

poljar mentioned this pull request Mar 23, 2026

Support Sqlite storage type matrix-org/matrix-sdk-crypto-wasm#168

Open

richvdh self-requested a review March 25, 2026 12:12

richvdh requested changes Mar 25, 2026

View reviewed changes

crates/matrix-sdk-sqlite/src/connection.rs Outdated Show resolved Hide resolved

crates/matrix-sdk-sqlite/src/connection/default.rs Show resolved Hide resolved

crates/matrix-sdk-sqlite/src/crypto_store.rs Outdated Show resolved Hide resolved

richvdh changed the title ~~Added wasm32-unknown-unknown compilation target support for matrix_sdk_sqlite~~ Add wasm32-unknown-unknown compilation target support for matrix_sdk_sqlite Mar 25, 2026

NoManColonies requested a review from a team as a code owner March 26, 2026 04:20

NoManColonies force-pushed the main branch 3 times, most recently from 5e409c1 to bf489aa Compare March 26, 2026 10:28

NoManColonies marked this pull request as draft March 27, 2026 19:58

NoManColonies force-pushed the main branch from 36759ed to ad559d8 Compare March 28, 2026 11:26

NoManColonies mentioned this pull request Mar 28, 2026

Configurable VFS for matrix_sdk_sqlite compiled to WASM targets #6365

Draft

3 tasks

NoManColonies marked this pull request as ready for review March 28, 2026 17:18

richvdh self-requested a review March 30, 2026 10:23

Geng Kongpop added 6 commits March 31, 2026 00:54

chore(changelogs): update change log

7bfe5a1

fix(sqlite): add js feature to support compiling sqlite to WASM target

746b2e8

chore(sqlite): fix lint warning under WASM target

5325493

refactor(sqlite): create SyncWrapper replacement struct for avoidin…

bb04434

…g conditional compilation

refactor(sqlite): move db fs setup to utils module

7d698fd

Geng Kongpop added 14 commits March 31, 2026 00:55

chore(lint): apply lint fix

946b397

refactor(vfs): avoid global installation of VFS

c7aff45

feat(test): make state store test run in WASM environment

bcd727e

feat(test): make media store test run in WASM environment

51f0837

feat(test): make event cache store test run in WASM environment

39f27c9

feat(test): make crypto store test run in WASM environment

1aac394

chore(lint): apply lint suggestion

0b9d6e4

feat(ci): configure xtask to run sqlite wasm test suites

cddebae

fix(ci): force SQLite wasm test to run in browser only

4ec20bf

chore(ci): allow dead code when certain features in disabled

281aca3

feat(ci): run matrix sdk wasm test with sqlite features enabled

4e1b849

fix(ci): missing testing feature

ec87f20

fix(test): incorrect database name passed when clear_data is true

fc742f3

chore(changelog): update sdk change log

7ed7f48

NoManColonies force-pushed the main branch from 675dd89 to 9d27165 Compare March 30, 2026 17:56

Geng Kongpop added 2 commits March 31, 2026 01:05

refactor(connection): split wasm implementation into a separate module

42108b1

chore: remove unintented changes from rebase conflict

4bd5f07

NoManColonies force-pushed the main branch from d92729b to 4bd5f07 Compare March 30, 2026 18:05

richvdh requested changes Mar 31, 2026

View reviewed changes

	//! An implementation of `deadpool` for `rusqlite` for usage
	//! An implementation of `deadpool::managed::Manager` for `rusqlite` for usage

		# [m]-sqlite is not supported on Node.js yet, due to
		# dedicated worker requirement for OPFS

Conversation

NoManColonies commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Notes for reviewers

Uh oh!

codecov bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codspeed-hq bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

richvdh left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

richvdh commented Mar 26, 2026

Uh oh!

NoManColonies commented Mar 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

richvdh left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

NoManColonies commented Mar 20, 2026 •

edited

Loading

codecov bot commented Mar 20, 2026 •

edited

Loading

codspeed-hq bot commented Mar 20, 2026 •

edited

Loading

richvdh left a comment •

edited

Loading

NoManColonies commented Mar 28, 2026 •

edited

Loading