WASI build: setjmp/longjmp error recovery works under standardized wasm EH (wasi-sdk 33 + wazero) — findings and patches from embedding pglite.wasi in Go

[moznion]

Summary

We embedded the REL_17_5_WASM-pglite WASI build (branch head 2194acf, built with the in-tree wasm-build.sh + portable-sdk 3.1.74.12.0) in a Go host under wazero, and got full PostgreSQL ERROR recovery working in the WASI flavor: erroring queries come back as normal wire-level errors with correct SQLSTATEs and the same instance keeps serving. Our stress gate — 120 mixed error/success iterations, aborted-transaction semantics (25P02 → ROLLBACK), a 1000-error storm on one instance, deep-recursion limits — passes 200 consecutive runs with real pgx clients.

Along the way we found and fixed several issues that we believe affect any consumer of the WASI build, not just ours. Everything below is maintained as a small patch series against 2194acf (build/patches/0002…0006 in https://github.com/moznion/wasipg, built reproducibly by build/build.sh); we'd be glad to turn any of it into PRs if there's interest.

1. Enabling sjlj (the headline)

The "sjlj exception handler off" state turns out to be three separate layers; with all three flipped, PG's own sigsetjmp recovery works out of the box:

1. Toolchain: the portable-sdk's hotfix/patch.h (force-included after user CFLAGS) stubs sigsetjmp to return 0 and siglongjmp to a no-op. Since PG 17's no-sigsetjmp fallback in c.h is Windows-only, these stubs are the only definition the build ever sees. We rewrite them to real setjmp/longjmp when a WASIPG_SJLJ macro is set (this has to happen in patch.h itself —a later -include cannot win the macro-redefinition race).
2. src/backend/utils/error/elog.c: the #if defined(__wasi__) ERROR → abort() in errfinish (patch 0002; also makes the non-sjlj abort emit the error report first, so hosts can at least see why).
3. pglite-wasm/pgl_sjlj.c: #if defined(__wasi__) compiles out the whole sigsetjmp block including the PG_exception_stack hookup (the code that prints "sjlj exception handler off"), so every ERROR is promoted to FATAL by pg_re_throw's NULL-stack branch (patch 0005).

Build flags: -mllvm -wasm-enable-sjlj -mllvm -wasm-use-legacy-eh=false plus -lsetjmp at link (patch 0004), compiled with wasi-sdk 33 (LLVM 22). The -wasm-use-legacy-eh=false part matters: LLVM's default legacy EH opcodes are rejected by runtimes that implement the standardized exnref proposal (wazero ≥ 1.12 in our case; the legacy opcodes are on their way out generally). wasi-sdk 25's LLVM 19 cannot emit exnref sjlj at all, hence the toolchain bump.

Measured cost of sjlj on the module: +0.2% size, no measurable boot or per-query latency change.

2. Silent shadow-stack overflow (worth fixing for every consumer)

The wasi-c wrapper hardcodes -z stack-size=131072 after user flags, so builds get a 128 KiB shadow stack that cannot be overridden from build scripts. Combined with the --no-stack-first layout, those 128 KiB sit directly above the data section — PostgreSQL's parser/planner recursion overflows into global data long before PG's max_stack_depth check (2 MB default) can trigger, silently corrupting arbitrary state. We saw intermittent traps at completely unrelated sites (relcache hash, ProcArray, ResourceOwner) until we raised the wrapper's stack size to 8 MiB; after that, deep recursion surfaces as PostgreSQL's clean, recoverable stack-depth/parser-depth ERROR. Recommendation: make the stack size overridable (or default it much higher for the PG build), or use --stack-first so overflow traps instead of corrupting.

3. freopen(stdin) in pgl_initdb breaks under wazero and newer wasi-libc

Two independent problems with the initdb boot-script replay (freopen(IDB_PIPE_BOOT, "r", stdin)):

• wazero's fd_renumber refuses stdio targets (it marks fds 0–2 as preopens), so wasi-libc's freopen (open + dup3) fails — and musl's failure path then closes stdin outright. The replay dies with input in flex scanner failed. (Reported separately to wazero: fd_renumber refuses stdio targets (marked as preopens), which breaks wasi-libc's freopen on stdin/stdout/stderr wazero/wazero#2518)
• wasi-libc 33 declares stdin as FILE *const, so the existing stdin = fdopen(saved_stdin, "r") restore no longer compiles.

Patch 0003 redirects at the fd level instead: close(0); open(...) lands the script on fd 0 via lowest-free allocation, no stream reassignment needed. Works under both wazero and wasmtime.

4. Wire-loop findings in interactive_one (WASI socket-file transport)

Driving the module hard with a real pgx client surfaced several issues in the per-tick wire loop (all addressed in patch 0006):

• The tick prologue is not idempotent, so hosts must not call interactive_one as a "poll" when they merely want pending output: the prologue resets MessageContext (freeing memory still referenced by a half-processed extended-protocol batch), invalidates the catalog snapshot, and consumes the send_ready_for_query flag. A host-scheduling-dependent number of poll ticks made identical runs diverge into traps, hangs and off-by-one reply pairing. We added a side-effect-free wasipg_flush export (emit the deferred ReadyForQuery, rename buffered reply bytes onto the reply file, touch nothing else) and drive real ticks only to consume input; something like it would be a great official addition to the ABI.
• The idle-tick block that real PostgresMain uses to send ReadyForQuery only cleared the flag — if the previous tick could not emit RFQ (auth in progress), the deferred RFQ was silently dropped.
• wire_flush skipped the PGS_OLOCK → PGS_OUT rename when the sockfiles flag was momentarily false while reply bytes were buffered; the next tick's fopen(OLOCK, "w") then truncated the stranded reply.
• Bytes left in the pq receive buffer by a partially-processed batch (error mid-pipeline) were never drained by later ticks — the tail, often the Sync, never executed and the client hung. Empty ticks now route into the wire loop when leftover pq bytes exist.
• The recovery sigsetjmp armed at the incoming: label, after io_init / snapshot invalidation / startup-auth processing — an ERROR raised in that window threw a longjmp no handler in that invocation had registered: uncaught exception, instance dead. The arming now happens at the top of the tick.

Two host-facing semantics that cost us time and may be worth documenting for binding authors:

• After error recovery, pq_comm_reset can leave libpq's send accounting in a state where a later flush re-emits an already-delivered reply; a host that trusts the byte stream blindly gets its request/reply pairing shifted by one. We guard host-side (each exchange is sealed at its ReadyForQuery).
• The auth handshake exchange legitimately ends without ReadyForQuery (e.g. AuthenticationMD5Password awaiting the client), so "wait for Z" is not a valid completion rule during startup.

Repro / artifacts

Everything is reproducible from a clean checkout of https://github.com/moznion/wasipg: ./build/build.sh builds the bundle from pinned sources (branch REL_17_5_WASM-pglite @ 2194acf, portable-sdk 3.1.74.12.0, wasi-sdk 33 overlay; every input sha256-pinned), and go test ./conformance/ runs the error-recovery suite with a real pgx client under wazero. Happy to rebase the patch series onto whatever branch is most useful, split it into PRs, or share more traces.

[StachowiakDawid]

1. Please don't paste large LLM's outputs clearly addressed to you as a different project's developer.
2. Did you check if those errors persist in Node.js WASI implementation? Try to import PGLite WASI build to Node.js and run it. If it runs, build minimum reproduction of broken error handling in current PGLite WASI build. Then it will be easier for people to verify patches provided by you or reproduce bugs that don't have patches yet.

[moznion]

@StachowiakDawid

Did you check if those errors persist in Node.js WASI implementation? Try to import PGLite WASI build to Node.js and run it. If it runs, build minimum reproduction of broken error handling in current PGLite WASI build. Then it will be easier for people to verify patches provided by you or reproduce bugs that don't have patches yet.

Yes, I've verified this against node's builtin WASI (node:wasi / uvwasi, node v24.18.0).
I wrote a dependency-free node host that drives pglite.wasi the same way the pglite-bindings gateway does: https://gist.github.com/moznion/501dd99be95f266f1dd2eda3fb446967

Results with a stock build of REL_17_5_WASM-pglite @ 2194acf:

$ node -v
v24.18.0
$ ./build-stock.sh # this builds REL_17_5_WASM-pglite
$ mkdir work-stock && tar xf stock-build/out/pglite-wasi.tar.xz -C work-stock
$ node pglite-node-wasi.mjs work-stock 'select 1+1 as OK' 'select 1/0' 'select 1+1 as SHOUL_STILL_ALIVE'
(node:60379) ExperimentalWarning: WASI is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
-> select 1+1 as OK
-> select 1/0
guest trapped: RuntimeError: unreachable

# 475:/tmp/pglite/base/.s.PGSQL.5432.in
# 547: PG_RE_THROW(ERROR : 0) ignored

As above, the broken error handling reproduces exactly like under wazero. Any erroring statement kills the instance.

The patched build, same host, same node: ErrorResponse with the right SQLSTATE, then ReadyForQuery, instance keeps serving:

$ git clone https://github.com/moznion/wasipg
$ (cd wasipg && ./build/build.sh)
$ mkdir work && tar xf wasipg/build/out/pglite-wasi.tar.xz -C work
$ node pglite-node-wasi.mjs work 'select 1+1 as OK' 'select 1/0' 'select 1+1 as SHOUL_STILL_ALIVE'
(node:60478) ExperimentalWarning: WASI is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
-> select 1+1 as OK
-> select 1/0
<! ERROR: division by zero (SQLSTATE 22012)
-> select 1+1 as SHOUL_STILL_ALIVE
instance survived all queries (1/3 returned an ERROR)

FYI:

• the freopen(stdin) issue does not reproduce under node (uvwasi allows fd_renumber onto stdio), so that one is wazero-specific.
• to build the stock module at all I had to drop --no-stack-first from pglite-wasm/build.sh: the lld 19 in the pinned portable-sdk doesn't know that flag, so the WASI build currently doesn't link as documented.