Sqlite Wasm build: build script, README, shell polyfill

[danleh]

Runs in JSC, SpiderMonkey shell, and D8. TODOs:

• Make compatible with jshell.
• Lower iteration counts / speed up benchmark.
• Get some data about noise / variance -> informs the lowering of runtime.
• Integrate into JetStream driver/runner (both cli.js and the browser one).
• Make sure the browser runner/version uses the same virtual file system, i.e., produces consistent performance results with the JS shell version (no OPFS!)

Sample output:

dlehmann@machine:~/JetStream/sqlite3$ ~/v8/v8/out/x64.release/d8 shell-runner.js 
Ignoring inability to install OPFS sqlite3_vfs: The OPFS sqlite3_vfs cannot run in the main thread because it requires Atomics.wait().
Calling main with argv:
  speedtest1,--singlethread,--nomemstat,--big-transactions,speedtest1.db
-- Speedtest1 for SQLite 3.47.0 2024-10-21 16:30:22 03a9703e27c44437c39363d0baf8
 100 - 50000 INSERTs into table with no index......................    0.077s
 110 - 50000 ordered INSERTS with one index/PK.....................    0.085s
 120 - 50000 unordered INSERTS with one index/PK...................    0.112s
 130 - 25 SELECTS, numeric BETWEEN, unindexed......................    0.059s
 140 - 10 SELECTS, LIKE, unindexed.................................    0.152s
 142 - 10 SELECTS w/ORDER BY, unindexed............................    0.211s
 145 - 10 SELECTS w/ORDER BY and LIMIT, unindexed..................    0.120s
 150 - CREATE INDEX five times.....................................    0.167s
 160 - 10000 SELECTS, numeric BETWEEN, indexed.....................    0.066s
 161 - 10000 SELECTS, numeric BETWEEN, PK..........................    0.063s
 170 - 10000 SELECTS, text BETWEEN, indexed........................    0.101s
 180 - 50000 INSERTS with three indexes............................    0.163s
 190 - DELETE and REFILL one table.................................    0.160s
 200 - VACUUM......................................................    0.200s
 210 - ALTER TABLE ADD COLUMN, and query...........................    0.004s
 230 - 10000 UPDATES, numeric BETWEEN, indexed.....................    0.086s
 240 - 50000 UPDATES of individual rows............................    0.080s
 250 - One big UPDATE of the whole 50000-row table.................    0.034s
 260 - Query added column after filling............................    0.003s
 270 - 10000 DELETEs, numeric BETWEEN, indexed.....................    0.154s
 280 - 50000 DELETEs of individual rows............................    0.097s
 290 - Refill two 50000-row tables using REPLACE...................    0.385s
 300 - Refill a 50000-row table using (b&1)==(a&1).................    0.183s
 310 - 10000 four-ways joins.......................................    0.181s
 320 - subquery in result set......................................    0.426s
 400 - 70000 REPLACE ops on an IPK.................................    0.132s
 410 - 70000 SELECTS on an IPK.....................................    0.044s
 500 - 70000 REPLACE on TEXT PK....................................    0.174s
 510 - 70000 SELECTS on a TEXT PK..................................    0.087s
 520 - 70000 SELECT DISTINCT.......................................    0.088s
 980 - PRAGMA integrity_check......................................    0.292s
 990 - ANALYZE.....................................................    0.044s
       TOTAL.......................................................    4.230s

[danleh]

Regarding test duration/workload size: There is a convenient --size option for the speedtest1 program, which I now set to 20 (default: 100), such that the benchmark takes <800ms. That would be acceptable for me (and lowering it even further would make me uneasy whether each individual sub-item still gets enough time and if that gives realistic results).

Regarding noise/variance: Running the benchmark 20 times with the reduced workload size in d8, jsc, and SpiderMonkey's js, I get the following results:

engine	mean(s)	std	std(%)
v8	0.6621	0.00734059	1.10868
jsc	0.7343	0.00593917	0.80882
js	0.8056	0.0278292	3.45447

1-3% standard deviation sounds fine to me.

[kmiller68]

Nice! Looks good so far!

One thing that might be worthwhile is to put the harness/driver hooks in their own file. I noticed when I tried to recompile some of the existing tests we couldn't just drop in emcc's new .js files because the harness was mixed in.

[danleh]

It now works in the CLI runner of JetStream and when running standalone via sqlite3/runner.js in shells. I still need to

• intercept one warning output/print, that originates internally in sqlite3, and
• make sure it works correctly (without OPFS) in the browser runner.

One thing that might be worthwhile is to put the harness/driver hooks in their own file. I noticed when I tried to recompile some of the existing tests we couldn't just drop in emcc's new .js files because the harness was mixed in.

Good point. The polyfills and runner are fully separated from the generated code.

This PR also currently includes/depends on #15.

[danleh]

@kmiller68 This is ready for your review from my side.

It's still using and scored with the "old" WasmBenchmark class, but we can eventually migrate it (after having merged it) to the same scoring as you do for TSF-wasm in #18. WDYT?

[kmiller68]

I guess for webpages they probably only need single threaded? Does it work without single threaded on the web or is that argument de facto required?

[danleh]

I copied this argument from the upstream speedtest1.html harness, which also uses --singlethread in the Web version of this benchmark. I don't see any observable difference with or without it, however. I have a minor preferences for leaving it as-is, to keep it consistent with the upstream benchmark harness. WDYT?

[kmiller68]

Overall looks good to me. Can we fix the one indentation issue first though?

[kmiller68]

This PR also currently includes/depends on #15.

Do you want to land that one first or do it here. I'm ok with either.

[danleh]

Overall looks good to me. Can we fix the one indentation issue first though?

Done.

Also renamed runner.js to benchmark.js as you proposed for consistency.

This PR also currently includes/depends on #15.

Do you want to land that one first or do it here. I'm ok with either.

Let's just land this here, I'll close #15 subsequently.

[danleh]

Ready from my side (sorry again for the delay.)

[kmiller68]

Looks good! Thanks.