feat: batch step transitions on by default, implemented on every World#3025
feat: batch step transitions on by default, implemented on every World#3025pranaygp wants to merge 17 commits into
Conversation
Collapse a durable step transition — complete step N, then create & start
the lone next inline step N+1 — into ONE atomic POST to the world's
`events/batch` endpoint, replacing the two serialized event writes
(step_completed(N) ~65-92ms + a lazy step_started(N+1) ~50-70ms in
production traces). Removes a full HTTPS round-trip per step; the create
claim now commits atomically with the previous completion.
- world (interfaces): optional `events.createBatch(runId, events, params)`
returning one EventResult per event plus an optional inline delta. Optional
so world-local / world-postgres stay valid and fall back to `create`.
- world-vercel: implement createBatch as 1..N back-to-back v4 frames, each
byte-identical to a single-event POST body (Content-Type
application/octet-stream, no envelope/sentinel — the server reads to EOF).
sinceCursor/stateUpdatedAt ride on the step_completed frame's meta;
vercelId per frame. Response is CBOR { results, eventsDelta? }. Retries
transient transport failures in-process (the batch is idempotent-on-retry).
- core: `WORKFLOW_BATCH_TRANSITIONS` (default OFF). In the sequential inline
path, defer a lone inline step's step_completed (from the 2nd step onward),
then at the next clean single-lazy-inline suspension issue the batch, seed
N+1 via a new executeStep `preStarted` param, and consume the batch
eventsDelta like pendingInlineDelta. NOT optimistic start — N+1's body runs
only after the batch returns.
Fallbacks restore today's behavior exactly: flag off, World without
createBatch, 404/405 (endpoint absent → disable batching for the invocation
+ flush the deferred write), parallel/hooks/waits/run-completion/first-step/
final-step. 409 (claim lost) / 410 (not running) / 412 (stale) abandon the
deferred completion and re-derive from a fresh replay without failing the run.
All new paths gate on `isBatchTransitionsEnabled()` / a pending deferral, so
the default-off path is byte-for-byte today's behavior.
Relates to vercel/workflow-server#646 (server endpoint). Independent of and
composable with the deferred-write mechanism in #2972.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: e80a06d The changes in this PR will be included in the next version bump. This PR includes changesets to release 20 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
✅ vercel-multi-region
❌ Some E2E test jobs failed:
Check the workflow run for details. |
📊 Workflow Benchmarkscommit Backend:
📜 Previous results (4)ff16390Tue, 21 Jul 2026 12:37:22 GMT · run logs
e0eab39Tue, 21 Jul 2026 11:48:45 GMT · run logs
f3bc16cTue, 21 Jul 2026 08:29:23 GMT · run logs
b4f5040Tue, 21 Jul 2026 07:55:20 GMT · run logs
Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
On an idempotent already-applied 200 from the batch endpoint (a concurrent or redelivered writer committed this exact step transition first), the server returns the current entities with no `stepCreated` on the step_started result — deliberately, so a non-committer neither double-bills nor re-runs the step body (workflow-server events.ts:4808-4824). The client was seeding `preStarted` from `results[last].step` unconditionally and running step N+1's body regardless, breaking the exactly-once guarantee the single-event lazy-start path enforces (a lost create-claim there surfaces as EntityConflictError -> `skipped` and never runs the body). Gate the inline continuation on `startedResult.stepCreated === true`. When absent, abandon the deferred completion and `reinvoke(0)` exactly like the 409/410 paths: the transition is already durable, so the fresh replay observes it and runs the body only if this invocation truly owns the claim (existing owned-recovery / inline-ownership logic). Adds a regression to runtime-batch-transitions.test.ts (body runs once on a fresh commit; zero times on already-applied, which nacks instead) plus body- execution counters. Found by the correctness review's adversarial repro. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Follow-up fix: gate
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
| ...request, | ||
| runId, | ||
| eventId: `evnt_synthetic_${request.correlationId ?? 'step'}`, | ||
| createdAt: new Date(), |
There was a problem hiding this comment.
The batch-transition replay pins the workflow VM clock to a wall-clock new Date() synthetic step_completed(N), so clock-dependent inter-step code bakes a non-deterministic timestamp into the durably-committed step_created(N+1), causing ReplayDivergenceError on later replays — and since WORKFLOW_BATCH_TRANSITIONS is now default-ON, this hits all deployments with a createBatch-capable World without opt-in.
…eview Point the world-vercel HTTP client at the workflow-server#646 preview deployment (head a588ed3d) so this PR's e2e/integration lanes exercise the real batched POST /api/v4/runs/:runId/events/batch backend before #646 is promoted to production. Probed pre-pin: POST /api/v4/runs/<fake>/events/batch returns 401 (auth), not a route-404 — the batch route is served by the preview. This is an intentional review-time pin. The CI lint rule requiring WORKFLOW_SERVER_URL_OVERRIDE to be empty on `main` will fail while this is present; that failure is the deliberate merge gate, not something to work around. Removal is a single-line revert to '' once #646 deploys to production. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ill switch Flip the batch step-transition flag to default-ON, mirroring the default-on + kill-switch shape of WORKFLOW_TURBO and WORKFLOW_RETURN_VALUE_STREAM. isBatchTransitionsEnabled() now returns true when the var is unset/empty; WORKFLOW_BATCH_TRANSITIONS=0 (or 'false', case-insensitive) is the emergency kill switch that restores the exact single-event two-POST path (step_completed then step_started), byte-identical to pre-feature behavior. The kill switch is TEMPORARY: it exists for emergency rollback during burn-in and is planned for removal once the batch path has production mileage. It is not a user-configurable feature. Test rework: - runtime-batch-transitions.test.ts: beforeEach now clears the var so every test runs on the true default-ON path; the old "flag OFF" test becomes a kill-switch test (=0 -> createBatch never called, two POSTs per step); the old "flag ON" test drops its explicit '1' and asserts the default directly; redundant '1' settings removed from the error-path tests. - runtime-batch-already-applied.adversarial.test.ts: beforeEach clears the var (default-ON) instead of setting '1'. - runtime-batch-attr-ordering.adversarial.test.ts unchanged: its in-test 0-vs-1 pairs are a deliberate off/on causal-ordering contrast. Core unit suite: 72 files, 1505 passed + 3 expected-fail under default-ON (matches the pre-flip baseline; zero regressions). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ansition world-local now implements the createBatch contract of workflow-server#646 so the default-ON batch path is genuinely exercised on the filesystem World (not silently falling back to single POSTs). The transition is "complete step N, then born-run step N+1", and the single-create path already has a battle-tested born-running fold (the lazy step_started that carries the step-creation input: it writes the step entity, a synthetic step_created event, transitions to running, and stamps stepCreated on the create-claim winner). createBatch composes that fold: it drives create() twice — step_completed(N) then a lazy born-running step_started(N+1) — so the durable state is structurally identical to the kill-switch (two-POST) path. A real fs-backed test asserts that equality directly. Atomicity is honest for a filesystem World: there is no cross-entity transaction, and the causal invariant completed(N) < created(N+1) requires completing N before claiming N+1, so strict all-or-nothing and causal ordering can't both hold on a plain FS. An already-applied peek makes re-delivery a no-op (writes nothing, omits stepCreated so the client re-derives from a fresh replay rather than double-running the body); the only residual non-atomic window cannot occur in the single-process dev model and would otherwise surface as the same EntityConflictError the single-event lost-claim path already handles. Tests (real fs-backed storage, no mocks): happy-path born-run + stepCreated; structural equality vs the two-POST kill-switch path; idempotent re-application (no new events, no stepCreated); malformed- batch rejection. Full world-local suite: 469 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ction world-postgres now implements the createBatch contract of workflow-server#646, so the default-ON batch path is genuinely exercised on the Postgres World. The whole transition [step_completed(N), step_created(N+1), step_started(N+1)] is applied in ONE drizzle.transaction — the same all-or-nothing guarantee the server provides over HTTP: any thrown error rolls the entire batch back. Implementation mirrors the single-create per-type logic but threads every read and write through the transaction handle `tx` (the module-level prepared statements run on a separate pooled connection and would not see the tx's uncommitted rows). A bornRunning set tracks which step_created frame won its create-claim; the matching step_started stamps stepCreated:true only for a fresh commit, and omits it on an already-applied retry / lost race so the non-committer re-derives from a fresh replay. Idempotent re-application takes the guarded-UPDATE-0-rows / onConflictDoNothing paths and writes nothing new. world-postgres does not enforce the optimistic stateUpdatedAt guard (neither does its single-create path), so it never raises 412; the client tolerates that. Tests (testcontainers Postgres, real transactions): happy-path born-run + stepCreated; structural equality vs the two-POST kill-switch path (step IDs normalized to roles since stepId is a global PK); idempotent re-application (no new events, no stepCreated); ALL-OR-NOTHING rollback (a failure mid-batch leaves N running, M absent, zero new events); empty/ missing-runId rejection. Full world-postgres suite: 153 passed. Also updates the Storage.events.createBatch interface doc: both world-local and world-postgres now implement it, so the batch path runs on every World rather than falling back. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add the WORKFLOW_BATCH_TRANSITIONS runtime variable to the Inline execution section of runtime-tuning.mdx, next to WORKFLOW_TURBO (its closest analog — a default-on optimization with a kill switch). Describes what the batch transition collapses, when it applies (non-turbo inline path, lone next step, no open hook/wait), and that setting 0/false restores the exact prior two-write path byte-for-byte. A warning callout states plainly that the flag is temporary — a burn-in rollback switch planned for removal, not a long-term user-configurable option. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…override Three getEventV4/createWorkflowRunEventV4 tests hardcoded the mock origin 'https://vercel-workflow.com' while their siblings in the same file already derive it from WORKFLOW_SERVER_URL_OVERRIDE || 'https://vercel-workflow.com'. With the review-time server override pinned, requests go to the preview origin but the mock was registered for the old one, so the interceptor missed and fetch failed. Align them with the sibling pattern so they pass with the override pinned AND after it is reverted to '' before merge. This is not a workaround for the empty-on-main lint gate (that gate stays failing by design); it just keeps the unit suite honest under the pin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…orlds) Bump the batch-step-transitions changeset from patch to minor across core, world, world-vercel, world-local, and world-postgres, and rewrite the summary: default-ON with a temporary WORKFLOW_TURBO-style kill switch, and events.createBatch now implemented on every World (Vercel endpoint, world-postgres single transaction, world-local guarded write sequence) rather than being an opt-in flag with single-write fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Converting to draft — upgrading in place to "suspension batch v2" (see the matching note on vercel/workflow-server#646). The client will collect the entire suspension output into one atomic batch (deferred completion + attrs/hooks/waits + parallel step claims + terminals) instead of only sequential step transitions — eliminating the attr-ordering flush class structurally and cutting parallel fan-outs to one round trip. Default-on + kill switch, per-World implementations, and the preview-pin CI setup all carry forward. Will re-mark ready when v2 lands against the updated server preview. 🤖 Generated with Claude Code |
Additive fields on the createBatch World interface for suspension-batch v2 (a whole suspension's event set committed in one conditional transaction): - CreateBatchParams.expectedRunVersion — per-run monotonic fence the runtime believes is current; a v2-capable World applies the whole batch conditionally on it (mismatch -> 412/409, writes nothing) and rejects pre-v2 runs that have no runVersion so the runtime falls back to the single-write path for them. - CreateBatchParams.batchId — idempotency key stable per suspension attempt; a batch whose id matches the run's lastBatchId resolves without re-writing. - BatchEventResult.runVersion — the run's version after the batch applied; the runtime stores it as the next expectedRunVersion. - BatchEventResult.lastBatchId — the batch id the World last applied; equals the request batchId on both a fresh apply and an idempotent already-applied resolution. All optional and absent from v1 responses, so v1 callers and Worlds that don't implement the fence are unaffected. This is the World-interface (logical) layer; the Vercel World's HTTP transport for the fence is finalized in the wire-encoding step against the v2 server contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…alse' kill switch Round-3 review low nits: - Track the two concurrent lost-race adversarial suites (world-local + world-postgres) that prove exactly-one `stepCreated` under genuine concurrency, not merely sequential redelivery — the B1 invariant the tracked storage-batch.test.ts does not exercise (its postgres pool is max:1, so its transactions never overlap). - Add an explicit WORKFLOW_BATCH_TRANSITIONS='false' kill-switch test and correct the comment that wrongly claimed the attr-ordering suite covered the 'false' word form (it only ever sets '0'). constants.ts disables on '0' || 'false'; both are now proven. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pure additive module: assembleSuspensionBatch orders one suspension's batchable components into the server's published v2 grammar (leading outcome first, terminal last, created+started adjacent) and budgets against the 100-item TransactWriteItems cap using the server's item arithmetic. attr_set and hook_created/hook_disposed are excluded by construction (physics: per-key attr updates and iad1-pinned hook entities cannot join a regional transaction). No live path calls this yet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wire the client's existing sequential batch transition to the committed v2 server contract (workflow-server 0d1b9848), the linchpin the worlds and collect-mode build on: - world: additive optional WorkflowRun.runVersion (the per-run monotonic fence; run_created seeds 0, absent on pre-v2 runs). - runtime: seed expectedRunVersion once per invocation from the loaded run; a run with no runVersion latches batchDisabledForRun (permanent by immutability — every invocation re-derives it). Gate batchTransitionCandidate and eligibleToDefer on it. - runtime: assemble the batch frames through assembleSuspensionBatch (identical [completed, created, started] for the sequential case) and carry the fence (expectedRunVersion + a fresh bat_<ulid> batchId) on createBatch; advance the local fence from result.runVersion after a successful batch (single-event writes never touch it). - runtime: apply-half maps 409 run-not-versioned to the PERMANENT per-run latch (flush the deferred completion on the single path, then reinvoke) — distinct from the transient 409 (suspension-batch-conflict), which the World surfaces as EntityConflictError/RunExpiredError and which keeps its abandon+reinvoke. B2 stays load-bearing: attr_set is still excluded from the batch (per the final contract) and attributeCount===0 remains in the candidate gate, so the flush-before-attr ordering is unchanged. Immediate (flag-off / pre-v2 / world without createBatch) paths are byte-identical. Tests: +3 v2 cases (fence carried & seeded 0; pre-v2 run latches batching off; run-not-versioned backstop flushes not abandons). runVersion:0 added to the two adversarial run mocks so they stay v2-eligible. Core unit suite 1516 pass + 3 expected-fail; typecheck + biome clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generalize world-local createBatch to the committed v2 contract: - run_created initializes runVersion:0 (both the direct and resilient-start paths) so a world-local run is v2-eligible; its absence marks a pre-v2 run. - world (schema): additive optional WorkflowRun.lastBatchId (a World's already-applied bookkeeping; the runtime never reads it). - createBatch(runId, events, params): when the caller supplies a batchId (the runtime always does for a v2 run) enforce the fence — reject a run with no runVersion as 409 run-not-versioned; short-circuit an already-applied batchId (lastBatchId match) to an idempotent no-write result with no stepCreated; reject a runVersion mismatch as EntityConflictError (the client abandons + reinvokes); otherwise apply the transition, then advance runVersion to expectedRunVersion+1 and record lastBatchId under the run lock. Return runVersion + lastBatchId. A fence-less caller (direct tests, legacy) keeps the exact v1 behavior. The durable step create-claim remains the hard exactly-once gate (honest FS-atomicity note preserved). Tests: +4 v2 fence cases (apply+advance+stamp; idempotent already-applied; version-mismatch aborts writing nothing; pre-v2 rejected). Full world-local suite 475 pass; typecheck + biome clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generalize world-postgres createBatch to the committed v2 contract, threading the fence through the existing single Postgres transaction: - schema + migration 0016: additive nullable run_version + last_batch_id columns on workflow_runs (NULL marks a pre-v2 run). run_created (direct + resilient start) seeds runVersion:0. - createBatch: when the caller supplies a batchId, SELECT ... FOR UPDATE the run row so the version check and advance are a genuine CAS inside the tx — reject a NULL run_version as 409 run-not-versioned; short-circuit an already-applied batchId (the per-frame loop already runs read-only, and the advance is skipped); reject a runVersion mismatch as EntityConflictError (rolls back the whole tx). On the fresh path advance run_version to expectedRunVersion+1 and record last_batch_id via a conditional UPDATE (0 rows -> conflict). Return runVersion + lastBatchId. Fence-less callers keep exact v1 behavior. - web-shared: add runVersion/lastBatchId to the attribute-panel display map (internal fence bookkeeping, rendered as null) since AttributeKey = keyof WorkflowRun now includes them. Tests: +4 v2 fence cases against testcontainers Postgres (apply+advance in-tx; idempotent already-applied echoes the version without re-advancing; version-mismatch rolls back writing nothing; pre-v2 rejected). Batch suites 11 pass; full-repo typecheck (41 tasks) + biome clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| // server reads them off the batch's "primary" (step_completed) frame, and a | ||
| // create+start-only batch has no completion to guard or diff against. | ||
| const isCompletion = data.eventType === 'step_completed'; | ||
| return { |
There was a problem hiding this comment.
[P0] The v2 runtime passes expectedRunVersion and batchId, but this adapter never copies either value from batchParams onto the primary frame. workflow-server#646 at 0d1b9848 requires both on frame 0, so every Vercel-world batch currently fails with 400 v4-missing-batch-fence. Please serialize both fields on the first event (not specifically step_completed, since valid v2 batches need not have a leading outcome) and add a wire test asserting their placement.
| const bodyBytes = new Uint8Array(await response.arrayBuffer()); | ||
| const decoded = | ||
| bodyBytes.byteLength > 0 | ||
| ? (decode(bodyBytes) as { |
There was a problem hiding this comment.
[P0] The current server response includes runVersion, but the decoder type and returned object drop it. Even after the request fence is serialized, the runtime will retain the old expectedRunVersion, so the next batch in the same invocation conflicts. Please decode/return runVersion through CreateEventBatchV4Result and BatchEventResult, with a regression covering two consecutive batches advancing 0 → 1 → 2.
| */ | ||
| export const WORKFLOW_SERVER_URL_OVERRIDE = ''; | ||
| export const WORKFLOW_SERVER_URL_OVERRIDE = | ||
| 'https://workflow-server-8sp9l2esp.vercel.sh'; |
There was a problem hiding this comment.
[P0] This preview is pinned to server commit a588ed3d, but workflow-server#646 is now at 0d1b9848 with the required v2 fence contract. Testing against the old v1 endpoint masks the missing expectedRunVersion/batchId request fields and dropped response runVersion. Please repin to the current server preview before treating cross-repo integration as green.
| err.waitCount === 0 && | ||
| err.attributeCount === 0 && | ||
| !batchOpenState.openHook && | ||
| !batchOpenState.openWait; |
There was a problem hiding this comment.
[P1] This gate means the live client still uses only the original single-inline-step triple, despite the server and assembleSuspensionBatch supporting fan-out, waits, hook_received, and terminal events. Under the existing counting convention this yields about N+6 requests, not N+1, because the first transition and final completion remain unbatched. A follow-up should collect the complete batchable suspension and use the existing budget/splitting logic; attr_set and hook_created can continue to force a pre-flush.
| err.waitCount === 0 && | ||
| !openHookWaitState.openHook && | ||
| !openHookWaitState.openWait && | ||
| (batchTransitionActive || inlineStepsExecuted >= 1); |
There was a problem hiding this comment.
[P1] Requiring inlineStepsExecuted >= 1 leaves step 1's completion and step 2's start as separate writes, losing one transition-level RTT on every multi-step workflow. The stated reason is preserving first-step telemetry, but the completion telemetry can ride on the leading frame of the same batch. Please either batch the first transition or document/measure why telemetry cannot be preserved while doing so.
| // `run_completed` stay on the existing path — flush the | ||
| // completion now, then re-replay so `run_completed` is | ||
| // written from the authoritative log exactly as today. | ||
| if (pendingBatchTransition) { |
There was a problem hiding this comment.
[P1] The v2 server already accepts [step_completed, run_completed], but this path flushes the final completion, replays again, and writes run_completed separately. Wiring the terminal event through assembleSuspensionBatch removes one server round trip plus this extra replay and is the clearest next optimization toward a N+1 transition model.
What
Batch step transitions, on by default. In the sequential inline runtime path, a step transition — completing step N, then creating + starting the lone next inline step N+1 — folds into one durable, atomic write instead of the two serialized event writes it did before:
step_completed(N)— ~65–92ms in production tracesstep_started(N+1)— ~50–70msWarm STSO was ~190ms/step (two HTTPS round-trips, each doing a run read → step-claim transact → patch → event insert) vs Temporal's ~47ms. Folding the transition into one conditional batch removes a full round-trip and commits the next step's create-claim atomically with the previous completion (strictly fewer intermediate crash states).
This is not optimistic start. Step N+1's body runs only after the batch (its durable claim) returns; the latency win is purely from collapsing two RTTs into one.
Default-on + temporary kill switch
WORKFLOW_BATCH_TRANSITIONSis on by default. Setting it to0(orfalse) is an emergency kill switch that restores the exact prior two-write path (step_completedthen a lazystep_started) byte-for-byte — mirroring the default-on + kill-switch shape ofWORKFLOW_TURBOandWORKFLOW_RETURN_VALUE_STREAM.How
@workflow/world(interfaces):events.createBatch(runId, events, params)returns oneEventResultper event in order plus an optional inline delta.@workflow/world-vercel:createBatchas 1..N back-to-back v4 frames, each byte-identical to a single-event POST body ([u32 meta_len][cbor meta][u32 body_len][body]),Content-Type: application/octet-stream, no envelope/sentinel (the server reads to EOF).sinceCursor/stateUpdatedAtride on thestep_completedframe's meta;vercelIdper frame. Response is CBOR{ results, eventsDelta? }. Transient transport failures retry in-process (the batch is idempotent-on-retry).@workflow/core: in the sequential inline path, defer a lone inline step'sstep_completed(from the 2nd step onward — the first step stays on the single-write path so its TTFS telemetry is measured as today), then at the next clean single-lazy-inline suspension issue the batch, seed N+1 viaexecuteStep'spreStartedparam, and consume the batcheventsDeltalikependingInlineDelta.Implemented on every World (real e2e coverage)
The batch path is no longer opt-in-by-World with a silent single-write fallback — every World implements
createBatch, so the default-on path is genuinely exercised in each World's own test suite:world-vercel— the/events/batchendpoint (above).world-postgres— the whole transition in onedrizzle.transaction(true all-or-nothing: any error rolls the batch back). Threads every read/write through the tx handle (the module-level prepared statements run on a separate pooled connection and wouldn't see uncommitted rows). AbornRunningset tracks the create-claim winner;stepCreated:trueis stamped only on a fresh born-running commit, omitted on already-applied/lost-race. Tested against a testcontainers Postgres, including a dedicated all-or-nothing rollback test.world-local— a per-run write sequence guarded by its on-disk exclusive claims. The transition is "complete N, then born-run N+1", and the single-createpath already has a battle-tested born-running fold (the lazystep_startedcarrying the create input), socreateBatchcomposes it:create(step_completed(N))then a lazy born-runningcreate(step_started(N+1)), making the durable state structurally identical to the kill-switch two-POST path (asserted directly against a real fs-backed storage). Atomicity is honest for a filesystem World: there's no cross-entity transaction, and the causal invariantcompleted(N) < created(N+1)requires completing N first, so strict all-or-nothing + causal ordering can't both hold on a plain FS; an already-applied peek makes redelivery a no-op, and the only residual window can't occur in the single-process dev model and would otherwise surface as the sameEntityConflictErrorlost-claim the single path already handles.Across all Worlds,
stepCreatedon the started result is the create-claim winner signal; an already-applied retry or lost race omits it so the non-committer re-derives from a fresh replay instead of double-running the body (matches the server,lib/data/events.ts:4808-4824).Deployment sequencing (required) + review-time server pin
POST /api/v4/runs/:runId/events/batch) must be deployed to production before this change ships in an@workflow/*SDK release. With the flag now default-on, a client that reaches a server without the endpoint would otherwise attempt one batch POST per invocation before falling back.404/405the runtime disables batching for the rest of that invocation, flushes the deferred completion, and falls back to single POSTs; a fresh delivery re-probes. This is deliberate: workflow-server is a stateless, heterogeneous fleet, so during the fix: include next config in web package #646 rollout a client's batch POSTs hit different server instances round-robin. Per-invocation re-probing opportunistically uses the endpoint on instances that already have it and degrades gracefully as the rollout completes; a per-process latch would stick pessimistically off for a warm instance's whole lifetime after a single early 404 from a not-yet-updated instance. The deployment-sequencing guarantee above avoids the window entirely; per-invocation re-probe is the graceful behavior during any overlap.WORKFLOW_SERVER_URL_OVERRIDE(packages/world-vercel/src/utils.ts) to the fix: include next config in web package #646 preview (https://workflow-server-8sp9l2esp.vercel.sh, server heada588ed3d) so the e2e/integration lanes run against the real batched backend before fix: include next config in web package #646 is in production. The CI lint rule that requires this constant to be empty onmainwill fail while the pin is present — that failure is the deliberate merge gate. Removal is a single-line revert to''once fix: include next config in web package #646 deploys to production. (Threeevents-v4unit tests that hardcoded the old mock origin were parameterized on the same override constant so the unit suite stays honest under the pin — correct with or without it.)Fallbacks (restore prior behavior exactly)
=0/false); parallel steps / hooks / waits / attribute writes / run-completion / non-inline steps / first step of the invocation / final step alone.With the kill switch thrown, all new paths are inert, so the disabled path is byte-for-byte the prior behavior (asserted by a test).
Correctness analysis
Reviewed adversarially against the server implementation (workflow-server#646). Both blockers found in review are fixed:
eventWasCreated:falseand nostepCreatedwhen a canceled transaction finds step N already completed and step N+1 already existing — regardless of who created N+1 (resolveStepTransitionBatchCancellation,lib/data/events.ts:4808-4824). The batch path previously seededpreStartedfromresults[last].stepunconditionally and ran N+1's body even when a concurrent/redelivered owner had won the claim (double-execution in the zombie/owned-recovery window; the single-event path would have skipped). Fix: gatepreStarted+ body execution onstartedResult.stepCreated === true; on a falsy signal, abandon and reconcile from a fresh replay. Regression:runtime-batch-already-applied.adversarial.test.ts+ a positive control inruntime-batch-transitions.test.ts.attr_set, preservingcompleted(N) < attr_set. Mechanism: a suspension carrying an attribute write is excluded frombatchTransitionCandidate(err.attributeCount === 0), sopendingBatchTransition && !batchTransitionCandidatefires the single-path flush ofcompleted(N)beforehandleSuspensionwritesattr_set. Without this, thehasAttributeEventsreplaycontinuerestarted the loop before the batch commit, strandingcompleted(N)and re-derivingattr_seteach iteration until a replay-budgetrun_failed(livelock + ordering inversion the single-write path can't produce). Regression:runtime-batch-attr-ordering.adversarial.test.tsasserts batch-on produces the identical durable event order to the disabled control for both the raced and un-raced shapes; verified to fail (livelock) with the one-line fix removed.Known follow-up (non-blocking)
{failedIndex, reason}→ in-placeskipped(perf). The server serializes{failedIndex, reason}on a 409; astep-claim-loston the started element could map to the in-placeskippedhandling instead of a fullreinvoke. Deferred: the batch is all-or-nothing, so astep-claim-lostalso leavescompleted(N)un-written — not a clean in-place substitution, and the safeabandon+reinvokealready handles it.Tests
world-vercel/events-batch.test.ts— wire contract (concatenated frames, no sentinel, octet-stream,sinceCursor/stateUpdatedAton thestep_completedframe,vercelIdper frame), nestedeventsDeltadecode, error mapping (409/410/412/404 no-retry).core/runtime-batch-transitions.test.ts— default (no env) folds the middle transition into one[completed, created, started]batch; kill switch (=0) asserts the exact two-POST-per-step pattern (nocreateBatch); world-lacks-createBatchand 404 disable and fall back; 409/410 abandon + nack without failing the run; exactly-once body counters.core/runtime-batch-already-applied.adversarial.test.ts— body does not run on an already-applied 200 (nostepCreated).core/runtime-batch-attr-ordering.adversarial.test.ts— batch-on produces the identical durable event order to the disabled control (raced + un-raced shapes).world-local/storage-batch.test.ts— real fs-backed: happy-path born-run +stepCreated; structural equality vs the two-POST kill-switch path; idempotent re-application (no new events, nostepCreated); malformed-batch rejection.world-postgres/test/storage-batch.test.ts— testcontainers Postgres: happy-path; structural equality vs two-POST (step ids normalized to roles, sincestepIdis a global PK); idempotent re-application; all-or-nothing rollback (a failure mid-batch leaves N running, M absent, zero new events); empty/missing-runId rejection.Suites green under default-on:
@workflow/core(1505 passed, 3 expected-fail),@workflow/world-vercel(305),@workflow/world-local(469),@workflow/world-postgres(153, testcontainers). Typecheck clean across all five packages. (Thee2e/*build/deploy tests need aDEPLOYMENT_URLand build toolchain — not runnable locally; they exercise the batch path in CI against the pinned #646 preview.)🤖 Generated with Claude Code