perf(core): retain workflow VM across inline steps#2990
perf(core): retain workflow VM across inline steps#2990NathanColosimo wants to merge 31 commits into
Conversation
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
✅ vercel-multi-region
|
📊 Workflow Benchmarkscommit Backend:
📜 Previous results (9)89494d1Mon, 20 Jul 2026 23:04:03 GMT · run logs
33f8e1aMon, 20 Jul 2026 21:03:00 GMT · run logs
3498706Mon, 20 Jul 2026 18:49:28 GMT · run logs
3498706Fri, 17 Jul 2026 23:54:22 GMT · run logs
e961466Fri, 17 Jul 2026 23:40:11 GMT · run logs
2b73676Fri, 17 Jul 2026 23:10:31 GMT · run logs
43788b8Fri, 17 Jul 2026 22:52:28 GMT · run logs
3bda890Fri, 17 Jul 2026 22:33:42 GMT · run logs
c6d6cb5Fri, 17 Jul 2026 22:02:43 GMT · run logs
Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
🦋 Changeset detectedLatest commit: 2e78951 The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
- A mixed step batch (one unsafe sibling input) now serializes every input through the ordinary VM path: a clone snapshotted before an unsafe sibling's serialization runs its getters could otherwise durably capture stale sibling state. - crypto.subtle.digest rejects SharedArrayBuffer-backed views with TypeError, matching WebCrypto's BufferSource contract.
devalue serializes Map/Set through the realm's iterator protocol and Date/RegExp/typed arrays through prototype getters, all of which workflow code can mutate — so their serialization is not provably passive and their bytes could differ between retained and cold modes. The fast path now accepts only primitives, plain objects, and plain arrays, which devalue traverses exclusively via own-property reads. Slot-bearing exotics decline even with a swapped prototype. The sandbox digest now reads view metadata (buffer/byteOffset/ byteLength) through captured intrinsic getters, so own properties shadowing them cannot change which bytes are hashed or bypass the SharedArrayBuffer rejection.
instanceof dispatch (Symbol.hasInstance via the constructor, Function.prototype, and Object.prototype), the class reducer's value.constructor walk, and devalue's Object/Array traversal all consult intrinsics workflow code could redefine — legally and deterministically — which would make the durable step input depend on WORKFLOW_RETAINED_VM (spoofed values serialize as e.g. Maps on the cold path but as plain clones on the retained path). Freeze Object/Array/Function (constructors and prototypes), the VM collection constructors, and every reducer-referenced global binding (absent ones pinned to undefined) right before the workflow bundle evaluates, so the retained-input equivalence holds by construction. Host-realm constructor escapes (e.g. TextEncoder.constructor) remain out of the determinism contract: code scheduling host timers was never deterministic under ordinary replay either; documented on canRetainWorkflowSession.
Typed-array constructors (and their shared %TypedArray% parent), the Date wrapper, and the session-local AbortController/AbortSignal/ Request/Response bindings were pinned but not frozen, so workflow code could still add Symbol.hasInstance statics that diverge reducer dispatch between the retained clone (host constructors) and ordinary VM serialization. Freeze every binding value that is not the shared host intrinsic; shared host objects are dispatched identically by both paths, so mutations there cannot cause mode divergence.
Replace structuredClone with an explicit deep copy into an SDK-private realm: clones previously inherited host prototypes, which workflow code can reach (e.g. via structuredClone's return values) and vandalize with Symbol.toStringTag or constructor overrides, shifting devalue's classification of the clone relative to the ordinary VM path. The pristine realm is unreachable by any user code, and the explicit copy serializes exactly what devalue traverses (own indices, own enumerable string props). Arrays also now decline own constructor properties, which the class reducer reads even when non-enumerable.
Host intrinsics are shared with the whole process and cannot be frozen, but workflow code can reach them (structuredClone results, exposed host classes) and install Symbol.hasInstance predicates that distinguish the original from its clone — or WORKFLOW_SERIALIZE statics on host Object/Array that the class reducer reads for host-prototype originals (hydrated step results). prepareRetainedStepInput now verifies, via own-descriptor reads only, that every host dispatch point is pristine and declines retention before any clone exists — so a spoofed predicate can never observe or capture a pristine-realm object.
Reducers dispatch on symbol tags (e.g. the workflow abort-signal markers) that are non-enumerable and dropped by the pristine-realm copy, so a tagged object would serialize as an abort descriptor on the cold path but as plain data on the retained path.
Hidden own keys of any kind — non-enumerable properties, accessors, symbols — can be observed by serialization dispatch (reducer probes like .signal, thenable checks, the class reducer) while the pristine clone drops them. With no hidden own keys, every probe on an accepted object resolves deterministically through validated data or pristine prototypes.
Symbol.hasInstance dispatch walks the constructor's prototype chain, so the frozen Date wrapper still exposed the unfrozen original VM Date it delegates statics to. Freeze each non-shared binding's full chain (stopping at host Function/Object prototypes) and verify host Object.prototype carries no added hasInstance on the detection side.
|
I'm re-running benchmarks one more time to see if the TTFS downsides hold |
pranaygp
left a comment
pranaygp
left a comment
There was a problem hiding this comment.
Trace-grounded context (durabench prod, SDK 5.0.0-beta.34, iad1, 2026-07-20): warm STSO on Workflow runs is ~150-270 ms/step (mean ~190) vs Temporal 45-50. Decomposing Datadog flow trace 2f55f784951f4eba33987bf54463b247 (run wrun_41KXZ0W77G0GJEPSCQEKGE823D), each warm step is ~65-92 ms of awaited step_started POST + ~50-70 ms of awaited step_completed POST + 28-78 ms of workflow.run full replay, plus 40-130 ms of process CPU per step. Because replay is O(journal length) per step, that slice grows O(steps^2) over a run. Retaining the VM removes the 28-78 ms replay slice (~30-40% of the warm gap) and most of the per-step CPU; the two serialized event POSTs (~120-160 ms) remain and are the next target. The complementary batched-transition work in vercel/workflow-server@pgp/batch-step-transitions + vercel/workflow@pgp/batch-transition-client collapses those two POSTs into one write, and the two changes together aim at Temporal-range STSO. The PR's own STSO deltas (-37/-50/-75% across the three windows) are consistent with removing the replay slice.
Of the two competing VM-retention PRs this is the stronger base, because it makes retention sound rather than fast-and-mostly-safe: the sandbox is hardened so a suspended VM provably cannot advance on host timing (the exact gap left open in #2966). The comments below are risk / edge-case / semver notes, not blockers.
Convergence: these two PRs implement the same idea and conflict; this one should be the base and #2966 closed in favor of it (or repurposed as a minimal default-off variant). #2966's genuine value -- the minimal EventsConsumer.resume mechanic and the createContext-called-once A/B test -- is already credited as absorbed here, so there is little left to graft.
Head-to-head durabench benchmark cells for this branch and #2966 are being added; numbers to follow as a comment.
| @@ -0,0 +1,6 @@ | |||
| --- | |||
| '@workflow/core': patch | |||
There was a problem hiding this comment.
Semver: this is marked patch, but the sandbox changes here ship unconditionally and are not reverted by WORKFLOW_RETAINED_VM=0 -- WeakRef, FinalizationRegistry, Atomics.waitAsync, and async WebAssembly.* are removed from the workflow global, crypto.subtle.digest switches to a node:crypto implementation, and Object/Array/Function (+ ~30 bindings) are frozen. Any deployed workflow that referenced one of those globals, or mutated a now-frozen intrinsic, changes behavior on upgrade. That is a runtime-observable breaking change to the workflow API surface, not a patch. Recommend a minor plus an explicit "removed globals" note -- ideally an errors-doc entry like the existing timeout-in-workflow / fetch-in-workflow ones -- so users get a clear signal instead of a bare TypeError.
| // - Dynamic `import()` settles within a microtask (rejected: no | ||
| // `importModuleDynamically`), so it needs no handling. | ||
| const intrinsics = g as unknown as Record<string, Record<string, unknown>>; | ||
| delete intrinsics.Atomics.waitAsync; |
There was a problem hiding this comment.
Worth stating explicitly here and in the runtime-tuning doc: the kill switch does not gate this block. WORKFLOW_RETAINED_VM=0 reverts the replay loop, but a workflow still loses these globals and gets the synchronous digest. That is the right call for making retention sound (these were never replay-safe), but it means the kill switch is not a full behavioral revert -- if an incident traces to the sandbox change rather than to retention itself, there is no env-only mitigation. Please make that explicit so on-call doesn't reach for WORKFLOW_RETAINED_VM=0 expecting the old sandbox back.
| // does not affect the workflow function or replay determinism. | ||
| // All SDK globals are installed; pin the serialization-consulted | ||
| // intrinsics before any workflow code can run. | ||
| freezeSerializationIntrinsics(vmGlobalThis); |
There was a problem hiding this comment.
freezeSerializationIntrinsics runs before every workflow body, on both the replay and retained paths, and is likewise ungated by the kill switch. Freezing Object/Array/Function + prototypes and pinning the binding list to non-writable/non-configurable will break workflow code that augments prototypes or reassigns these globals (polyfills, Object.defineProperty(Array.prototype, ...) shims, test instrumentation). Most workflows won't, but it is a real compatibility edge for a default-on change. Two asks: (1) confirm it is exercised on both the turbo and non-turbo entry paths (the benchmark step/stream scenarios stay in turbo, so the retained-loop tests should not be the only coverage of the freeze), and (2) same semver/doc point as the changeset.
| * workflow nondeterministic under ordinary replay too, and is not defended | ||
| * here. | ||
| */ | ||
| function canRetainWorkflowSession( |
There was a problem hiding this comment.
The retention gate looks right -- demoting on hooks/waits/attributes/hook-disposals/aborts and on any open hook/wait in history closes the concurrent-wake paths, and pairing it with the permanent per-session replay fallback is the correct posture. Two interactions I'd want an explicit test for before this rides default-on:
WORKFLOW_MAX_INLINE_STEPS: confirm a retained session still counts against the inline-step cap within one invocation. The retained VM advances without a fresh replay per step, so the accounting path differs from the loop this cap was written against -- a test that a retained run stops inlining at the cap (rather than running unbounded steps in-process) would lock that down.- feat(core): opt-in deferred step_completed write (WORKFLOW_ASYNC_STEP_COMPLETED) #2972 synthesized-events deferral: if both ship, a retained consumer that appends only newly-durable events could observe a different event set than a cold replay that re-synthesizes deferred events. A combined-flag integration test would keep the two features from interacting into a divergence that only surfaces in production.
| return Object.getOwnPropertyDescriptor(target, key) !== undefined; | ||
| } | ||
|
|
||
| function isHostDispatchPristine(): boolean { |
There was a problem hiding this comment.
The conservative failure mode here is the right design -- any doubt declines to replay, so this can't corrupt a run, and the all-or-nothing batch handling in handleSuspension is a good catch. Two smaller notes: (1) isHostDispatchPristine runs a ~35-entry host-global descriptor scan on every retainable suspension; almost certainly negligible next to the POSTs, but since a goal of this PR is cutting per-step CPU it's worth confirming it doesn't show up in the numbers. (2) This module plus freezeSerializationIntrinsics is a large, subtle surface whose correctness is coupled to devalue/reducer internals -- a pointer comment from the reducer/serialization code back to these invariants would save the next person who changes serialization from silently breaking the equivalence this relies on.
|
Head-to-head durabench benchmark (follows up the review): this branch vs #2966 vs
Takeaways: (1) −30% warm STSO and −83% per-step CPU vs baseline, with the win growing with step count (replay was O(journal)/step); (2) #2990 and #2966 are statistically indistinguishable on latency — the decision between them is purely the correctness/architecture question from the reviews; (3) the remaining ~120-135ms/step is the two serialized event POSTs, which retention can't touch — that's the batch-transition track ( 🤖 Generated with Claude Code |
pranaygp
left a comment
pranaygp
left a comment
There was a problem hiding this comment.
Production trace validation — VM retention confirmed working; converging on this PR
Follow-up to my earlier review. We're converging the inline-replay work onto this branch, so I deployed this exact commit (89494d10c, published as SDK 5.0.0-beta.35-89494d1) as a durabench Workflow host in iad1 and drove real runs through it, then pulled the Datadog APM traces to confirm the optimization actually fires and is observable. It does, cleanly.
Sweep: psweep-1784618537794-c4c10680-4bbf-4d6a-a73d-6575653f1fe5 (retainvm-0mb cell = this branch; beta-0mb = v5 beta baseline, no retention). Traces read from service:durabench-workflow-host, keyed on @workflow.run.id.
1. Retention is real and observable
The new workflow.execution.mode attribute is present on every workflow.run span and labels the first execution replay, every subsequent inline step retained:
| Config | exec spans | replay | retained | retained dur (median) | events.count growth in-run |
demotions |
|---|---|---|---|---|---|---|
| 5 steps | 6 | 1 | 5 | ~2 ms | 5 → 17 | 0 |
| 120 steps | 121 | 1 | 120 | 1.99 ms (p95 ~4 ms) | 5 → 362 (72×) | 0 |
| 1020 steps | 1021 | 2 | 1019 | 1.92 ms (quartile means 2.6/2.8/2.1/2.5) | 65 → 3062 (47×) | 0 |
2. Replay is eliminated — O(1) per step, confirmed at scale
Retained execution stays flat at ~2 ms median while the journal grows up to 72× within a single run. There is no per-step cost growth — the O(steps²) replay term is gone. On the beta baseline the equivalent workflow.run span is ~105 ms median (and carries no execution.mode attribute, confirming it's this-PR-only).
3. End-to-end warm STSO: 147 ms/step vs 247 ms/step (−40%)
Driver-measured step-to-step overhead: retainvm-120 = 147 ms/step vs beta-120 = 247 ms/step. And it's flat 120 → 1020 steps (147 → 140 ms/step), so the quadratic term is genuinely removed, not just shifted.
4. MAX_INLINE_STEPS / redelivery interaction is correct
The 1020-step run crosses one invocation boundary: 2 replays total — one at events.count=0 (initial cold execution) and one at events.count≈2606 (a single redelivery that rebuilds the VM from the journal), then resumes retained for the rest. So retention costs one rebuild-replay per invocation, not per step — exactly Temporal's sticky-execution model. This directly answers the earlier scrutiny about WORKFLOW_MAX_INLINE_STEPS: it holds.
5. What's left is not this PR's job
The remaining ~145 ms/step decomposes almost entirely into the two serialized event POSTs (step_started mean 62 ms + step_completed mean 67 ms ≈ 129 ms). The retained VM slice (~2 ms) is at its floor. The rest is the complementary batch-transition track (vercel/workflow-server branch pgp/batch-step-transitions + vercel/workflow branch pgp/batch-transition-client), which collapses those two POSTs into one transition write. The two changes compose: retention removes the CPU/replay slice, batching removes the network slice.
Verdict: this is the convergence base. The optimization works, is sound (per my earlier review of the sandbox quiescence), and is properly instrumented. Two small o11y refinements inline — neither blocking.
Reviewers can reproduce from any run in the sweep; each durabench run page emits a Datadog APM deeplink. Example (120-step): service:durabench-workflow-host @workflow.run.id:wrun_41KY1RVQ9J0GJWXV14Y512VTRH.
| ); | ||
|
|
||
| /** Whether workflow execution starts with replay or resumes a retained VM */ | ||
| export const WorkflowExecutionMode = SemanticConvention<'replay' | 'retained'>( |
There was a problem hiding this comment.
Validated in production — this attribute is the thing that made the whole optimization auditable from traces alone, so thank you for adding it. Two small refinements now that I've stared at real traces:
-
replayconflates two very different events. In a 1020-step run I seemode=replayat bothevents.count=0(initial cold execution) andevents.count≈2606(a redelivery that rebuilds the VM from the journal). Onlyworkflow.events.countdisambiguates them, and they have wildly different cost profiles (the redelivery replay is the one expensive O(journal) rebuild per invocation). Consider either a third value (e.g.cold|redelivery|retained) or a doc note thatevents.countmust be read alongside this. -
A retention demotion is invisible. When a retained session can't extend and falls back to a full replay (
workflow.ts:259), that execution is labeledreplaywith no signal that retention was attempted and failed. In prod that's indistinguishable from a normal redelivery. A companion attribute on the fallback path would make retention regressions observable — see the note atworkflow.ts:259.
| case 'resume': { | ||
| session = request.session; | ||
| const resumed = session.resume(request.events); | ||
| if (resumed.type === 'replay') return resumed; |
There was a problem hiding this comment.
This is the silent-demotion path. When resume() returns replay (stale session / diverged prefix), we correctly fall back to a full replay — but the resulting executeWorkflow span will read mode=replay (line 236) with nothing indicating retention was tried and rejected here.
On the pure-sequential timing workload this never fires — 0 demotions across 5/120/1020-step runs, so the fallback is currently unexercised by production telemetry. That's reassuring for correctness but means a real-world demotion (impure args, hook/wait-bearing workflow, prefix divergence) would show up only as an ordinary replay, with no counter or reason to alert on. Suggest setting something like workflow.execution.retention.demoted=true + a reason on the fallback execution so a retention regression is a queryable signal rather than a silent revert to baseline latency. Non-blocking; just closes the observability loop the execution.mode attribute opens.
… (v2) Serialize step inputs for retained boundaries through the one ordinary pipeline (original value, workflow global) instead of cloning into a pristine realm and serializing under the host global. With a single serialization event shared by every mode, durable bytes structurally cannot depend on WORKFLOW_RETAINED_VM; the only property retention needs is that serialization executes no workflow code, established by: - the passive walker (descriptor-only, unchanged in spirit), now also accepting Map/Set/Date/typed arrays/ArrayBuffer — the common built-in step arguments — via prototype-identity checks - vm/serialization-pins.ts: the 10 prototype members serialization executes for those built-ins (measured empirically), captured at context creation and identity-verified at each retained boundary; the 'touches only pinned members' test instruments every member and locks the list against serde drift - host-realm instances (hydrated step results) accepted without member verification: host members run host code, which cannot touch retained VM state Deletes the pristine clone realm, the host-dispatch pristineness checks, and the batch clone bookkeeping.
… (v3) Review found the pin approach's structural hole: the class reducer READS value.constructor through Map.prototype — a data property when pristine (so member instrumentation never listed it), but executable the moment workflow code redefines it as a getter. Pinning what serialization executes misses what it reads. Freeze the accepted built-ins' prototypes wholesale (Map/Set/Date + iterator prototypes, %TypedArray% + subclass prototypes, ArrayBuffer): reads and executes are both immutable, and a patch attempt now throws loudly at the patch site instead of silently degrading. Deletes vm/serialization-pins.ts; the walker requires Object.isFrozen on the realm prototype (also covering realms where the freeze never ran). Also restores the host-dispatch pristineness check the v2 cut lost: workflow code can reach shared host constructors (exposed classes, structuredClone results) and plant workflow-realm Symbol.hasInstance hooks or WORKFLOW_SERIALIZE statics that reducers would execute during retained serialization. Host-realm built-in instances decline for the same reason; host-realm plain data (hydrated results) stays retainable.
- Capture Map/Set forEach and the %TypedArray% buffer getter as module- load primordials: the checker previously invoked live host methods that workflow code can reach (structuredClone(new Map()).constructor) and replace with delegating workflow-realm closures. - Typed arrays must have one of the realm's real frozen subclass prototypes by identity — 'frozen and chains to %TypedArray%' admitted manufactured frozen hostile prototypes with delegating buffer getters.
…ializer statics - The walker resolved Object.getOwnPropertyDescriptor, Reflect.ownKeys, Array.isArray, Number/String helpers, and Object.getPrototypeOf/isFrozen from live host globals workflow code can reach and replace; all are now module-load captures, so the checker can never execute a planted delegate. - The class reducer reads cls[WORKFLOW_SERIALIZE]/cls.classId as inherited Gets, so isHostDispatchPristine now also verifies host Function.prototype and Object.prototype carry no serializer statics. Generic replacement of shared host statics (Object.keys, Array.from, …) via realm escape remains the documented host-reachability boundary, tracked by the realm-local intrinsics follow-up.
- Suspension signals capture ctx.suspensionGeneration when scheduled and no-op if the session resumed past that boundary. The harmful interleaving was already unreachable (queue items are deleted on consume, completion writes state synchronously, nextTick precedes timers) — the token turns those ordering facts into an explicit invariant. - The BigInt reducer calls .toString() on primitives from host code, which resolves on host BigInt.prototype: its identity joins the host dispatch check, and the VM BigInt.prototype is frozen besides.
Resolves runtime.ts around the executeWorkflow refactor: main's runWorkflow gained worldCapabilities, now threaded through WorkflowSessionOptions into the session context.
3 participants
Summary
crypto.subtle.digest, noAtomics.waitAsync/async-WebAssembly/WeakRef/FinalizationRegistryWORKFLOW_RETAINED_VM=0kill switch (default on) restores replay-from-scratch on every iterationworkflow.runspans withworkflow.execution.mode=replay|retainedfor direct production decompositionrunWorkflowas the one-shot compatibility APICombines #2984 (retained-session architecture) with the best tactics from #2966 (env kill switch, loop-level single-VM A/B test), plus review-driven sandbox hardening. Supersedes both.
Design
executeWorkflowowns the invocation-local VM behind two discriminated unions: a request (replay|resume) and a result (completed|suspended|replay). Overloads guarantee at the type level that a fresh replay can never request another replay — only resuming a stale or diverged session can.The session itself is a small state machine:
running,suspended,failed,replay, orcompleted. The runtime retains a session only for a pure step boundary —stepCount > 0, zero hooks/waits/attributes/hook-disposals/aborts, and no open hook or wait anywhere in the event history — since hooks and waits can wake concurrent invocations and attributes require replay.Unconditional VM quiescence
Retention is only sound if a suspended VM cannot make progress the event log cannot replay (found by review: a workflow racing host-timed async against a step could advance while the inline step ran and commit behavior no replay reconstructs). Rather than tracking such APIs, the sandbox now has none:
crypto.subtle.digestcomputes synchronously vianode:crypto(createHash— byte-identical values, verified against WebCrypto for SHA-1/256/384/512; stable and undeprecated through Node 26), so its promise settles on a deterministic microtask. Digest-using workflows stay retainable, and digest race timing becomes deterministic and identical across live, retained, and replay execution.Atomics.waitAsync(a wall-clock timer viaSharedArrayBuffer), the asyncWebAssemblycompilation entry points,WeakRef, andFinalizationRegistryare deleted from the sandbox — wall-clock timing and GC observation are unreplayable with or without retention. Syncnew WebAssembly.Module()/Instance()remain. Dynamicimport()settles within a microtask.A suspended VM is therefore parked exclusively on log-driven promises: its state is a pure function of the consumed event prefix — exactly what a replay rebuilds.
Serialization passivity (found and shaped by ~19 review rounds)
Step-argument serialization runs once per step (at suspension handling, never during replay), so any side effect it has on the VM exists in the live timeline but in no replayed one. Serialization always runs through the single ordinary pipeline — original value, workflow global — so the durable bytes structurally cannot depend on
WORKFLOW_RETAINED_VM; the only property retention needs is that serialization executes no workflow code. Established by four layers:retained-step-input.ts): descriptor-only inspection of step args — getters/proxies/functions/symbols/hidden keys/custom classes decline; plain data plusMap/Set/Date/typed arrays/ArrayBuffer(workflow-realm, frozen prototypes) pass. The checker itself runs exclusively on module-load-captured primordials.freezeSerializationIntrinsics):Object/Array/Function+ prototypes, the accepted built-ins' prototypes (incl. iterator prototypes and all typed-array subclass prototypes),BigInt.prototype, and every serialization-referenced global binding (+ prototype chains; absent bindings pinned toundefined). Members the serializer executes and members it merely reads (constructor) are both immutable; a workflow patch attempt throws at the patch site.isHostDispatchPristine): the shared host constructors/prototypes serialization consults cannot be frozen process-wide, but workflow code can reach them (exposed classes,structuredCloneresults) — so ~40 own-descriptor reads per retained batch verify no plantedSymbol.hasInstance, serializer statics (incl. inherited via hostFunction.prototype/Object.prototype), orBigInt.prototype.toString. Dirt declines retention for the boundary.Unsupported inputs serialize identically and the session demotes to replay for that one boundary (side effects land in a VM about to be discarded — exactly pre-retention semantics); retention resumes at the next clean boundary. A generation token additionally invalidates suspension timers queued before a resume.
Known residual (deliberate): workflow code that escapes to the host realm via exposed host-class constructor chains (e.g.
TextEncoder.constructor→ host timers, or replacing shared host statics likeObject.keys) breaks the whole process deterministically for every mode and was never replayable — that root cause requires realm-local sandbox intrinsics and is tracked as a follow-up.Validation
pnpm --filter @workflow/core typecheckpnpm --filter @workflow/core buildWORKFLOW_RETAINED_VM=0): 72 files, 1,541 passed, 3 expected failures eachBenchmark
STSO results land in the sticky benchmark comment below (the benchmark workflow runs against this PR's preview deployment).
For reference, #2984 measured the same architecture (on top of #2980, now merged) at:
Docs Preview
Behind deployment protection (Vercel team access required).
WORKFLOW_RETAINED_VMWORKFLOW_SERIALIZE— hook purity constraints