feat(world-postgres): add optional executor invocation for hooks

[shalabhc]

1. Changes to the World API

Add optional request/response delivery for run inputs:

world.capabilities?.invoke
world.invoke?(runId, payload, { idempotencyKey?, timeoutMs? }): Promise<unknown>

The existing createQueueHandler callback now returns Promise<unknown>. World
delivers invocation-mode messages through that same callback:

{ runId, invoke: true, requestId, input }

Core inspects the input, awaits its event write, and returns a value. World
owns delivering/storing that value for the original caller. There is no exported
Invocation type, metadata.invocations, public mailbox API or respond()
callback. The input loop is private to Postgres World. Core shares a run's
execution/admission activity between normal wake and input calls, so input
delivery does not start a competing replay while a step is waiting.

Only normal wake returns interpret { timeoutSeconds } as queue control. In
invocation mode it is response data. In-tree adapters have the corresponding
return-value guards; only Postgres advertises invocation sending in this PR.

Hooks are the first caller: supported resumeHook() payloads use invoke when the
capability is enabled. Legacy payloads and other Worlds retain the existing path.
The ordering is event-log write → handler return → World response storage;
there is no new acquisition, exchange or atomic-commit API. Invoke failures do
not fall back to direct producer-side event writes.

2. Postgres implementation and Graphile concurrency

Opt-in: after migrations, set WORKFLOW_POSTGRES_INVOKE=1 or pass
enableInvoke: true to createWorld(). Default is off. Apply migrations before
running the upgraded Postgres World even with invoke disabled: hook deduplication
and zero-retention cleanup also use the new columns.

The mailbox stores an input/fingerprint and eventual result under (runId, requestId). Input insertion and Graphile wake enqueue share a backend-private
transaction. Every eligible invoke enqueues a wake, including result retries;
redundant wakes still check durable run state rather than treating an empty
mailbox as proof that all committed events were replayed.

With the default job prefix:

Graphile task	Named queue	Concurrency
workflow_flows_executor	workflow_flows:<runId>:executor	One active executor job per run across workers
workflow_flows	Unset for step jobs and health checks	Remains parallel

The overall queueConcurrency remains 50 per process by default, not 1.
Different runs use different named queues. The executor's World wrapper services
pending inputs while the normal SDK handler is awaiting work, and stores each
input handler's returned value. The Graphile job stays unacknowledged for that
executor lifetime. A later executor may use another process and replay.

Executor delivery checks

The executor task checks its actual Graphile queue and forwards job/worker/attempt
metadata in private headers. The HTTP receiver checks the active task, exact run
queue, worker and attempt against graphile_worker.jobs before starting mailbox
service. Caller-supplied headers cannot override this metadata.

Updated workers/receivers durably transfer legacy unmarked orchestration into the
executor lane before acknowledging it. Steps and health checks never drain the
mailbox. These are entry checks, not continuous journal fencing of stale handlers.

Notification-driven delivery and response waiting

Both waiting points use LISTEN/NOTIFY with one lazy dedicated listener per World.
Input notifications commit with input/wake insertion; result notifications commit
with the separate response update. Notifications carry hashed identifiers and
always lead to authoritative table reads. Read revisions and subscription/reconnect
invalidation close the read-then-wait race. A 1-second fallback and reconnect
backoff cover missed signals/unavailable LISTEN. Degradation/restoration is logged
once per transition to stderr without connection details or payloads.

Inputs are read in pages of 32. Default response timeout is 30 seconds; encoded
inputs/results are limited to 1 MiB each. Closing the World cancels waits and
closes the listener. LISTEN needs a session-capable connection; otherwise fallback
reads preserve progress. Graphile still gets every invocation wake.

Example: one hook

resumeHook(token, value)
  -> World.invoke: insert H1 + enqueue E1 + notify

E1 claims the run's executor queue and calls the flow route
  -> Postgres wrapper reads H1
  -> SDK handler({invoke:true, requestId:H1, input})
  -> SDK validates and writes hook_received with durable resume identity
  <- SDK returns accepted
  -> Postgres wrapper stores result and notifies caller
  <- caller rereads result and resolves

The caller waits for H1's result, not for E1 or the whole workflow to finish.

Example: two hooks queued for the same run

invoke(H1) -> input H1 + wake E1 --+
                                +-> workflow_flows:<runId>:executor
invoke(H2) -> input H2 + wake E2 --+

E1 active: World delivers H1 to SDK, stores its returned result
           World delivers H2 to SDK, stores its returned result
E2 waits: no second executor job for this run starts while E1 is active
E1 exits: E2 runs and may find no additional work

If H2 arrives after E1 exits, E2 processes it instead. An already-active executor
can also consume both inputs before their extra wakes run. Different runs and
step jobs remain concurrent.

Review fixes

• Duplicate hook events: Postgres implements the existing hookResumeDedup
  capability. A unique (runId, resumeId) event identity and validated payload
  digest make the event write idempotent. Retry converges even after disposal or
  normal run completion, and changed contents under one identity are rejected.
  Event and response writes remain separate; the fix does not introduce a public
  transaction API. Separate logical calls with identical payloads stay distinct.
• Zero retention: purge now clears invocation input/result/fingerprint data
  and event resume digests. Expiry tombstones settle callers with
  INVOCATION_DATA_EXPIRED (410). Mailbox writers lock/recheck run lifecycle so
  late writes cannot restore purged data. Migration backfills already-expired or
  terminal zero-retention runs from earlier previews.
• Notification diagnostics: bounded degradation/restoration logs, with tests
  ensuring repeated failed reconnects do not spam or expose raw error details.
• Stale-handler coverage: a real Graphile claim-revocation characterization
  test confirms the entry guard does not fence a previously admitted writer.
  Full journal fencing remains the explicitly deferred limitation below.
• Concurrency/footprint: three parallel waves with up to 24 invokes each (72 inputs)
  check one shared producer listener, expected mailbox row growth, drained
  executor queues, and listener release after close. The fixture records result
  read counts; it is not a production throughput benchmark.

Validation

• 303 tests passed: core runtime/retained-VM/hook/handler-return suites, shared
  queue tests, local/Vercel queue compatibility and module-state checks.
• 264 tests passed: Postgres executor/queue/notification suites, real
  Postgres/Graphile invocation cases, storage, retention, run creation and status
  waiting. Includes Node and QuickJS, inline self-hook delivery, response-loss
  retry, post-disposal/completion dedup, zero-retention/late-write races,
  migration backfill, listener reconnect, reclaim and concurrency fixtures,
  including the updated HTTP deadline/error-metadata and stream-EOF regressions.
• Affected packages built successfully. Builders and simulator also typechecked
  after building their dependencies. Changeset and diff validation passed;
  Biome reports warnings, no errors.
• Merged current main, preserving its native HTTP delivery/deadline controls,
  queue error metadata, status-list handling and stream-EOF fixes.
• Integration uses a real HTTP host and Graphile runners in one test process;
  these are not multi-host process-kill or production load measurements.

Remaining limitations

• Graphile job serialization is not continuous fencing of a handler that keeps
  running after claim revocation. No new ownership protocol is introduced.
• Input admission runs while steps wait; VM continuation still follows existing
  replay boundaries. Long bodies can delay serialized wakes and occupy slots.
• Matching upgraded workers and queue configuration are required. Named queues
  do not route incompatible code versions; old binaries do not enforce new checks.
• General result/per-run-queue cleanup remains unimplemented outside explicit
  zero retention. Expiry can occur before a caller reads its response, even if
  the hook event already committed; the caller then receives the expiry error.
• Event deduplication is not exactly-once execution of arbitrary step side effects.

Docs Preview

• Postgres World — experimental invocation delivery
• Building a World — invocation delivery

Preview base URL from the vercel[bot] project row; team authentication is
required. Section anchors match the documentation headings.

[changeset-bot]

🦋 Changeset detected

Latest commit: ba49f37

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 20 packages

Name	Type
@workflow/world	Minor
@workflow/core	Minor
@workflow/world-postgres	Minor
@workflow/world-local	Patch
@workflow/world-vercel	Patch
@workflow/cli	Patch
@workflow/vitest	Patch
@workflow/web-shared	Patch
@workflow/web	Patch
@workflow/world-testing	Patch
@workflow/builders	Patch
@workflow/next	Patch
@workflow/nitro	Patch
workflow	Minor
@workflow/astro	Patch
@workflow/nest	Patch
@workflow/rollup	Patch
@workflow/sveltekit	Patch
@workflow/vite	Patch
@workflow/nuxt	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
example-nextjs-workflow-turbopack	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
example-nextjs-workflow-webpack	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
example-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-astro-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-express-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-fastify-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-hono-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-nestjs-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-nitro-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-nuxt-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-python-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-sveltekit-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-tanstack-start-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workbench-vite-workflow	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workflow-docs	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workflow-swc-playground	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workflow-tarballs	Ready	Preview, v0	Sep 15, 2026 1:38am UTC
workflow-web	Ready	Preview, v0	Sep 15, 2026 1:38am UTC

[github-actions]

🧪 E2E Test Results

✅ All tests passed

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

• sleepWinsRaceWorkflow (tanstack-start · local-dev / local / quickjs)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

• run-pickup-stall · hookCleanupTestWorkflow - hook token reuse after workflow completion (nextjs-webpack) · at 01:45:15Z · abandoned wrun_01M2HBTA31SCQFV6PDZKZYSBG4

E2E Test Summary

Summary

	Passed	Failed	Skipped	Total
✅ ▲ Vercel Production	3662	0	685	4347
✅ 💻 Local Development	3998	0	510	4508
✅ 📦 Local Production	3998	0	510	4508
✅ 🐘 Local Postgres	3998	0	510	4508
✅ 🪟 Windows	160	0	1	161
✅ 🌐 Cross-language Conformance	68	0	74	142
✅ vercel-http-transport	823	0	143	966
✅ vercel-multi-region	27	0	0	27
✅ vercel-ws-transport	557	0	87	644
Total	17291	0	2520	19811

Details by Category

✅ ▲ Vercel Production

App	Passed	Failed	Skipped
✅ astro-node	133	0	28
✅ astro-quickjs	133	0	28
✅ example-node	133	0	28
✅ example-quickjs	133	0	28
✅ express-node	133	0	28
✅ express-quickjs	133	0	28
✅ fastify-node	133	0	28
✅ fastify-quickjs	133	0	28
✅ hono-node	133	0	28
✅ hono-quickjs	133	0	28
✅ nest-node	133	0	28
✅ nest-quickjs	133	0	28
✅ nextjs-turbopack-node	158	0	3
✅ nextjs-turbopack-quickjs	158	0	3
✅ nextjs-webpack-node	158	0	3
✅ nextjs-webpack-quickjs	158	0	3
✅ nitro-node	133	0	28
✅ nitro-quickjs	133	0	28
✅ nuxt-node	133	0	28
✅ nuxt-quickjs	133	0	28
✅ python-node	66	0	95
✅ sveltekit-node	152	0	9
✅ sveltekit-quickjs	152	0	9
✅ tanstack-start-node	133	0	28
✅ tanstack-start-quickjs	133	0	28
✅ vite-node	133	0	28
✅ vite-quickjs	133	0	28

✅ 💻 Local Development

App	Passed	Failed	Skipped
✅ astro-stable-node	134	0	27
✅ astro-stable-quickjs	134	0	27
✅ express-stable-node	134	0	27
✅ express-stable-quickjs	134	0	27
✅ fastify-stable-node	134	0	27
✅ fastify-stable-quickjs	134	0	27
✅ hono-stable-node	134	0	27
✅ hono-stable-quickjs	134	0	27
✅ nest-stable-node	134	0	27
✅ nest-stable-quickjs	134	0	27
✅ nextjs-turbopack-canary-node	160	0	1
✅ nextjs-turbopack-canary-quickjs	160	0	1
✅ nextjs-turbopack-stable-node	160	0	1
✅ nextjs-turbopack-stable-quickjs	160	0	1
✅ nextjs-webpack-canary-node	160	0	1
✅ nextjs-webpack-canary-quickjs	160	0	1
✅ nextjs-webpack-stable-node	160	0	1
✅ nextjs-webpack-stable-quickjs	160	0	1
✅ nitro-stable-node	134	0	27
✅ nitro-stable-quickjs	134	0	27
✅ nuxt-stable-node	134	0	27
✅ nuxt-stable-quickjs	134	0	27
✅ sveltekit-stable-node	153	0	8
✅ sveltekit-stable-quickjs	153	0	8
✅ tanstack-start-node	134	0	27
✅ tanstack-start-quickjs	134	0	27
✅ vite-stable-node	134	0	27
✅ vite-stable-quickjs	134	0	27

✅ 📦 Local Production

App	Passed	Failed	Skipped
✅ astro-stable-node	134	0	27
✅ astro-stable-quickjs	134	0	27
✅ express-stable-node	134	0	27
✅ express-stable-quickjs	134	0	27
✅ fastify-stable-node	134	0	27
✅ fastify-stable-quickjs	134	0	27
✅ hono-stable-node	134	0	27
✅ hono-stable-quickjs	134	0	27
✅ nest-stable-node	134	0	27
✅ nest-stable-quickjs	134	0	27
✅ nextjs-turbopack-canary-node	160	0	1
✅ nextjs-turbopack-canary-quickjs	160	0	1
✅ nextjs-turbopack-stable-node	160	0	1
✅ nextjs-turbopack-stable-quickjs	160	0	1
✅ nextjs-webpack-canary-node	160	0	1
✅ nextjs-webpack-canary-quickjs	160	0	1
✅ nextjs-webpack-stable-node	160	0	1
✅ nextjs-webpack-stable-quickjs	160	0	1
✅ nitro-stable-node	134	0	27
✅ nitro-stable-quickjs	134	0	27
✅ nuxt-stable-node	134	0	27
✅ nuxt-stable-quickjs	134	0	27
✅ sveltekit-stable-node	153	0	8
✅ sveltekit-stable-quickjs	153	0	8
✅ tanstack-start-node	134	0	27
✅ tanstack-start-quickjs	134	0	27
✅ vite-stable-node	134	0	27
✅ vite-stable-quickjs	134	0	27

✅ 🐘 Local Postgres

App	Passed	Failed	Skipped
✅ astro-stable-node	134	0	27
✅ astro-stable-quickjs	134	0	27
✅ express-stable-node	134	0	27
✅ express-stable-quickjs	134	0	27
✅ fastify-stable-node	134	0	27
✅ fastify-stable-quickjs	134	0	27
✅ hono-stable-node	134	0	27
✅ hono-stable-quickjs	134	0	27
✅ nest-stable-node	134	0	27
✅ nest-stable-quickjs	134	0	27
✅ nextjs-turbopack-canary-node	160	0	1
✅ nextjs-turbopack-canary-quickjs	160	0	1
✅ nextjs-turbopack-stable-node	160	0	1
✅ nextjs-turbopack-stable-quickjs	160	0	1
✅ nextjs-webpack-canary-node	160	0	1
✅ nextjs-webpack-canary-quickjs	160	0	1
✅ nextjs-webpack-stable-node	160	0	1
✅ nextjs-webpack-stable-quickjs	160	0	1
✅ nitro-stable-node	134	0	27
✅ nitro-stable-quickjs	134	0	27
✅ nuxt-stable-node	134	0	27
✅ nuxt-stable-quickjs	134	0	27
✅ sveltekit-stable-node	153	0	8
✅ sveltekit-stable-quickjs	153	0	8
✅ tanstack-start-node	134	0	27
✅ tanstack-start-quickjs	134	0	27
✅ vite-stable-node	134	0	27
✅ vite-stable-quickjs	134	0	27

✅ 🪟 Windows

App	Passed	Failed	Skipped
✅ nextjs-turbopack-node	160	0	1

✅ 🌐 Cross-language Conformance

App	Passed	Failed	Skipped
✅ python	68	0	74

✅ vercel-http-transport

App	Passed	Failed	Skipped
✅ example	133	0	28
✅ express	133	0	28
✅ hono	133	0	28
✅ nextjs-turbopack	158	0	3
✅ nitro	133	0	28
✅ vite	133	0	28

✅ vercel-multi-region

App	Passed	Failed	Skipped
✅ nextjs-turbopack	27	0	0

✅ vercel-ws-transport

App	Passed	Failed	Skipped
✅ example	133	0	28
✅ express	133	0	28
✅ nextjs-turbopack	158	0	3
✅ vite	133	0	28

---

📋 View full workflow run

[github-actions]

📊 Workflow Benchmarks

commit ba49f37 · Tue, 15 Sep 2026 01:53:21 GMT · run logs

Backend: vercel · app: nextjs-turbopack

Metric	Scenario	Best (ms)	P75 (ms)	P90 (ms)	P99 (ms)	Samples
TTFS	step	1577 (+19%) 🔻	1681 🔴 (+17%) 🔻	1731 🔴 (+18%) 🔻	1883 🔴 (+17%) 🔻	30
TTFS	stream	1563 (+714%) 🔻	1637 🔴 (+15%)	1670 🔴 (+14%)	1767 🔴 (+17%) 🔻	30
TTFS	hook + stream	728 (+27%) 🔻	2118 🔴 (+15%) 🔻	2264 🔴 (+16%) 🔻	6839 🔴 (+197%) 🔻	30
Fan-out TTFS	Promise.all(100 steps)	468 (+3.3%)	901 (+33%) 🔻	912 (-51%) 💚	2172 (+17%) 🔻	10
Fan-out TTLS	Promise.all(100 steps)	2216 (+32%) 🔻	2835 (-13%)	5460 (+26%) 🔻	8982 (+14%)	10
STSO	1020 steps (inline)	108 (-14%)	146 (-8.2%)	162 (-10%)	210 (-0.9%)	1019
WO	1020 steps	147611 (-7.1%)	147611 (-7.1%)	147611 (-7.1%)	147611 (-7.1%)	1
CRTT	first chunk (pooled)	62 (-32%) 💚	101 (-32%) 💚	298 (+45%) 🔻	3993 (+1699%) 🔻	28

Streams

Scenario	CRTT 1st	p75	p90	p99	CDV max	iters
paced control (100/s, 60B)	85.5 (-41%)	232 (-42%)	392 (-34%)	601 (-27%)	114 (-60%)	10
size sweep (100/s, 160B-12KB)	93 (-30%)	198 (-50%)	354 (-33%)	3965 (+325%)	168 (-36%)	10
replay gateway-gpt-5.4-nano-2000t (1x)	67 (-41%)	138 (-48%)	198 (-50%)	648 (+9%)	235 (-42%)	3
replay eve-gpt-5.6-sol-2000t (1x)	96.5 (-41%)	139 (-46%)	168 (-50%)	373 (-51%)	270 (-61%)	2
replay eve-gpt-5.6-sol-2000t (2x)	71 (-44%)	180 (-61%)	246 (-62%)	431 (-64%)	245 (-43%)	3

📈 STSO distribution vs main (inline / queue-hop histograms)

1020 steps (inline)

Cumulative STSO time: main 158739ms → this run 147427ms (Δ -11312ms, -7%)

100-150 ms  ███████████████░░░░░░░░┃  main 512  this 823  +311
150-200 ms  ████┃█████████            main 477  this 181  -296
200-250 ms  ┃                         main  26  this  10   -16
250-300 ms  ┃                         main   2  this   1    -1
300-350 ms  ┃                         main   0  this   2    +2
350-400 ms  ┃                         main   0  this   2    +2
400-450 ms  ┃                         main   2  this   0    -2

📈 CRTT drill-down vs main (RTT distributions & profiles)

variant  RTT 1ms→5s+             avg         p50         p90           p99     n
control  ······▃█▂▁···  145.3 (-46%)  124 (-44%)  392 (-34%)    601 (-27%)  3000
sweep    ······▂█▂·▁▁·    386 (+35%)  129 (-44%)  354 (-33%)  3965 (+325%)  3000
gw 1x    ·····▁▅█▁▁···  124.2 (-38%)  109 (-38%)  198 (-50%)     648 (+9%)  5295
eve 1x   ·····▁▅█▁▁···  119.2 (-44%)  106 (-41%)  168 (-50%)    373 (-51%)  5186
eve 2x   ·····▁▂█▂▁···  152.2 (-53%)  139 (-55%)  246 (-62%)    431 (-64%)  7779

RTT over stream progress (avg per tenth of stream, bars scaled min→max):

control  ██▄▂▁▁▄▂▄▃  117–189ms
sweep    █▇▆▆▅▄▃▂▂▁  251–522ms
gw 1x    █▁▂▃▁▂▂▂▂▁  111–178ms
eve 1x   ▃▁▃▅▃▇▆█▃▁  105–139ms
eve 2x   █▂▂▂▁▁▅█▆▃  128–189ms

RTT by chunk size (avg per log size bin, ~160B → ~12KB serialized, bars scaled min→max):

sweep  ██▆▂▂▁▄  382–390ms

Delivery jitter over stream progress (avg positive CDV per tenth of stream, bars scaled min→max):

control  █▂▄▂▁▃▃▁▆▂  39–56ms
sweep    ▁▃▃▇▆█▃▁▆▂  50–74ms
gw 1x    █▄▄▄▄▅▆▇▃▁  30–44ms
eve 1x   █▅▁▆▇▇▆█▄▃  20–25ms
eve 2x   █▄▂▅▄▄▂▁▂▅  20–35ms

📜 Previous results (2)

51cff62

Mon, 14 Sep 2026 23:12:00 GMT · run logs

vercel / nextjs-turbopack

Metric	Scenario	Best (ms)	P75 (ms)	P90 (ms)	P99 (ms)	Samples
TTFS	step	206 (-84%) 💚	1668 🔴 (+16%) 🔻	1951 🔴 (+33%) 🔻	2061 🔴 (+28%) 🔻	30
TTFS	stream	189 (-1.6%)	1719 🔴 (+20%) 🔻	1746 🔴 (+19%) 🔻	1987 🔴 (+32%) 🔻	30
TTFS	hook + stream	2026 (+253%) 🔻	2370 🔴 (+29%) 🔻	2455 🔴 (+25%) 🔻	2634 🔴 (+14%)	30
Fan-out TTFS	Promise.all(100 steps)	466 (+2.9%)	2055 (+204%) 🔻	2160 (+17%) 🔻	2323 (+25%) 🔻	10
Fan-out TTLS	Promise.all(100 steps)	1708 (+1.8%)	4566 (+40%) 🔻	6903 (+59%) 🔻	8757 (+11%)	10
STSO	1020 steps (inline)	106 (-16%) 💚	162 (+1.9%)	181 (+0.6%)	284 (+34%) 🔻	1019
WO	1020 steps	162719 (+2.4%)	162719 (+2.4%)	162719 (+2.4%)	162719 (+2.4%)	1
CRTT	first chunk (pooled)	70 (-23%) 💚	114 (-23%) 💚	167 (-19%) 💚	258 (+16%) 🔻	28

Streams

Scenario	CRTT 1st	p75	p90	p99	CDV max	iters
paced control (100/s, 60B)	98 (-32%)	182 (-55%)	254 (-57%)	917 (+11%)	170 (-41%)	10
size sweep (100/s, 160B-12KB)	102 (-23%)	419 (+5%)	546 (+3%)	808 (-13%)	300 (+15%)	10
replay gateway-gpt-5.4-nano-2000t (1x)	91 (-19%)	190 (-29%)	246 (-37%)	534 (-10%)	404 (±0%)	3
replay eve-gpt-5.6-sol-2000t (1x)	103 (-37%)	177 (-31%)	270 (-20%)	568 (-25%)	573 (-17%)	2
replay eve-gpt-5.6-sol-2000t (2x)	97 (-24%)	287 (-38%)	421 (-36%)	953 (-21%)	354 (-18%)	3

b9a614c

Mon, 14 Sep 2026 21:32:00 GMT · run logs

vercel / nextjs-turbopack

Metric	Scenario	Best (ms)	P75 (ms)	P90 (ms)	P99 (ms)	Samples
TTFS	step	279 (+27%) 🔻	1691 🔴 (-4.9%)	1747 🔴 (-8.8%)	1917 🔴 (-8.7%)	30
TTFS	stream	192 (-15%)	1652 🔴 (-2.1%)	1704 🔴 (-1.8%)	1745 🔴 (-5.6%)	30
TTFS	hook + stream	1957 (+193%) 🔻	2062 🔴 (±0%)	2129 🔴 (+0.7%)	2247 🔴 (-12%)	30
Fan-out TTFS	Promise.all(100 steps)	433 (-16%) 💚	809 (+7.3%)	1025 (-52%) 💚	2409 (+1.5%)	10
Fan-out TTLS	Promise.all(100 steps)	1795 (+28%) 🔻	3151 (-21%) 💚	4193 (-9.7%)	9170 (+8.5%)	10
STSO	1020 steps (inline)	109 (-17%) 💚	141 (-14%)	159 (-14%)	221 (-15%)	1019
WO	1020 steps	140909 (-13%)	140909 (-13%)	140909 (-13%)	140909 (-13%)	1
CRTT	first chunk (pooled)	61 (-10%)	95 (-25%) 💚	102 (-31%) 💚	484 (+34%) 🔻	28

Streams

Scenario	CRTT 1st	p75	p90	p99	CDV max	iters
paced control (100/s, 60B)	79 (-25%)	134 (-47%)	311 (-10%)	488 (-13%)	178 (-12%)	10
size sweep (100/s, 160B-12KB)	85.5 (-12%)	201 (-22%)	301 (-43%)	791 (-17%)	202 (-34%)	10
replay gateway-gpt-5.4-nano-2000t (1x)	92 (-23%)	155 (-81%)	215 (-92%)	561 (-87%)	300 (-14%)	3
replay eve-gpt-5.6-sol-2000t (1x)	281 (+110%)	151 (-33%)	187 (-41%)	346 (-49%)	367 (-45%)	2
replay eve-gpt-5.6-sol-2000t (2x)	82 (-25%)	185 (-56%)	255 (-54%)	461 (-52%)	249 (-60%)	3

ℹ️ Metric definitions & methodology

_{Streams: first-chunk RTT (the stream-open path, before any buffering/backpressure), CRTT percentiles, and worst delivery stall (CDV max). Cells are medians across iterations; per-run values in the artifacts. No 🔴/🟢 marks until targets attach.}

_{The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.}

_{The collapsed CRTT drill-down: per-variant RTT histograms (fixed log bins, · = empty) and mean RTT/positive-CDV profile lines over stream progress and chunk size. Histograms, avgs, and profiles merge exactly across runs; p50–p99 are percentile-of-percentiles. Per-index rows live in the artifacts.}

_{Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.}

_{Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · CRTT: chunk round-trip time (per-chunk write → read latency, one clock domain: deployment → stream backend → same deployment) · CDV: chunk delay variation / delivery jitter (inter-arrival gap minus inter-write gap per seq-adjacent pair; skew-free; the row is each run's MAX positive value, so one stall moves it)}

_{Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out · paced control (100/s, 60B): the control: 300 tiny (~60B) deltas metronome-paced at 100/s — zero workload structure, so it reads the transport floor and flush cadence, and disambiguates transport-wide vs workload-specific when a replay row moves · size sweep (100/s, 160B-12KB): same pacing as the control with deltas padded in rotation across seven log-spaced sizes (~160B–12KB) — rotation decouples size from stream position, so it isolates whether chunk size causes latency · replay gateway-gpt-5.4-nano-2000t (1x): raw provider SSE cadence captured at the AI gateway boundary (gpt-5.4-nano, the most popular gateway model; per-token deltas p50 208B = the modal production chunk size), replayed exactly as measured — the typical customer's workload; its CDV is the typical customer's real delivery jitter · replay eve-gpt-5.6-sol-2000t (1x): a captured eve turn (gpt-5.6-sol, the most-used demanding eve model; ~2000 output tokens = production p50 turn length) replayed exactly as measured — eve's envelope protocol re-ships the cumulative message so sizes ramp 142B→13KB; the demanding outlier tenant's reality · replay eve-gpt-5.6-sol-2000t (2x): the same eve capture at 2x — the headroom/stress row; real fast-tier models emit the same chunk sizes at proportionally higher rate, so time compression is a faithful speed model · first chunk (pooled): every run's seq-0 RTT pooled across all stream scenarios — the first chunk precedes any workload differentiation, so pooling samples one shared stream-open path with exact percentiles}

_{Replay cadences (semantic sha256) — eve-gpt-5.6-sol-2000t eaf22f5946e7c61f3c65c7006d550df180cfabd4e706254a09f22aec0cfb420d · gateway-gpt-5.4-nano-2000t 6f24ac518b6b83ff1d0e85a5fe78230db192716d66a7fc6b2fe022752001d041}

_{🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600}

_{All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies; CRTT inside the workflow (excludes the api.vercel.com read path).}

_{Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.}

[github-actions]

Sim World

Simulated world deterministic testing for races. Traces

🟠 world-sim scenario book — 1 fail of 41 total

fence=per-spec

scenario	outcome	events	virt	replay	violations
✅ smoke-no-steps	completed	3	0ms	ok	0
✅ smoke-one-step	completed	6	0ms	ok	0
✅ hook-at-step-started	completed	12	0ms	ok	0
✅ hook-at-step-completed	completed	12	0ms	ok	0
✅ hook-at-hook-created	completed	12	0ms	ok	0
✅ deadline-hook-wins	completed	7	1.0h	ok	0
✅ deadline-expires	completed	7	1.0h	ok	0
✅ long-sleep	completed	11	30.0d	ok	0
✅ hook-never-arrives	stalled	3	0ms	skipped	0
✅ step-retries-twice	completed	10	2.0s	ok	0
✅ parallel-steps	completed	9	0ms	ok	0
✅ hook-on-execution-state	completed	12	0ms	ok	0
✅ peek-hook-before-branch	completed	12	0ms	ok	0
✅ peek-hook-after-branch	completed	12	0ms	ok	0
✅ peek-hook-at-registration	completed	12	0ms	ok	0
✅ race-hook-before-probe	completed	12	0ms	ok	0
✅ race-hook-after-probe	completed	12	0ms	ok	0
✅ race-duplicate-delivery	completed	13	0ms	ok	0
✅ attr-hook-before-step	completed	11	0ms	ok	0
✅ attr-hook-after-step	completed	11	0ms	ok	0
✅ attr-from-step-body	completed	13	0ms	ok	0
✅ fork-hook-after-timeout	completed	14	1.0m	ok	0
✅ fork-hook-before-timeout	completed	14	1.0m	ok	0
✅ count-hook-after-timeout	completed	17	1.0m	ok	0
✅ count-hook-before-timeout	completed	20	1.0m	ok	0
✅ stale-read-step-count-fork	completed	20	1.0m	ok	0
✅ stale-read-equal-step-counts	completed	14	1.0m	ok	0
✅ step-vs-step-fork	completed	12	0ms	ok	0
✅ step-vs-step-fork-fenced	completed	12	0ms	ok	0
✅ fence-catches-benign-direction	completed	12	5ms	ok	0
✅ in-flight-before-decision	completed	17	1.0m	ok	0
❌ in-flight-before-decision-counted	completed	17	1.0m	ok	0
✅ in-flight-after-decision	completed	19	2.0m	ok	0
✅ stale-read-step-count-fork-fenced	completed	20	1.0m	ok	0
✅ fork-hook-wins	completed	13	1.0m	ok	0
✅ fork-timeout-wins	completed	13	1.0m	ok	0
✅ unclaimed-payload-under-fork	completed	17	1.0m	ok	0
✅ claimed-payload-under-fork	completed	17	1.0m	ok	0
✅ writers-independent-step-bodies	completed	12	0ms	ok	0
✅ writers-scripted-tempo	completed	12	0ms	ok	0
✅ cancel-mid-step	cancelled	7	0ms	skipped	0

Full trace: world-sim.txt

[github-actions]

Framework	Flow route	Step reg.	Framework output
hono	250.7 KiB (±0)	93.2 KiB (+200 B)	1.89 MiB (+2.4 KiB)
nextjs-turbopack	257.2 KiB (±0)	426 B (±0)	899.3 KiB (+1.5 KiB)

About these numbers

Sizes are gzip; parentheses show the change against main.
Flow route and Step reg. gate this job, on raw bytes rather than the gzip shown, at max(2%, 50.0 KiB). Framework output is informational.

ba49f37 · run

[VaguelySerious]

AI review: blocking issues found

[VaguelySerious]

AI Review: Blocking

Redelivery after events.create() succeeds but respond() fails appends the same hook input again because invocation.id is not used to deduplicate the event. I added a crash-boundary test; it observed two hook_received events for one invocation ID. This can execute a single resume twice. Persist an invocation identity with the event or otherwise make redelivery idempotent.

[shalabhc]

Fixed in e3b9a6b using the existing resumeId/resumePayloadDigest API, without adding an atomic World commit method. Postgres validates the digest and resolves a prior event identity before lifecycle rejection. Core also recognizes prior acceptance after hook disposal/run completion. The real-DB test simulates the missing-response boundary, retries the same ID, verifies exactly one hook_received, and completes the actual workflow. Separate IDs with identical payloads remain distinct; changed contents under one ID are rejected.

[VaguelySerious]

AI Review: Blocking

Invocation payloads remain intact when a $retention: 0 run finishes. The existing purge clears all other user-data columns, but this new table is neither included nor lifecycle-linked. This violates the run’s retention contract and retains potentially sensitive hook payloads indefinitely. Add invocation data to the transactional purge and cover it with a retention test.

[shalabhc]

Fixed in e3b9a6b. Zero-retention purge now clears invocation payloads/results/fingerprints and event resume digests, retaining an expiry tombstone so callers receive INVOCATION_DATA_EXPIRED (410). Mailbox inserts and response writes lock/recheck run lifecycle, preventing late writes from restoring purged data. Migration 0021 also backfills already-expired or terminal zero-retention runs from earlier previews. Tests cover the existing retention fixture, pending callers, late writes racing the purge lock, and migration backfill.

[VaguelySerious]

AI Review: Note

This admission check happens only before the HTTP handler runs. If Graphile aborts or reclaims the job while that handler continues, a replacement executor can be admitted while the old handler still writes events, so the claimed per-run serialization no longer holds at the journal boundary. The limitation is documented, but there is no test modeling an admitted handler continuing through reclaim; please add one or fence subsequent writes.

[shalabhc]

Added the requested characterization coverage in e3b9a6b: a real Graphile job is admitted, its claim is revoked through forceUnlockWorkers, and the held HTTP handler is then allowed to write an event. The test explicitly confirms the remaining unfenced-write behavior rather than claiming it is safe. Continuous journal fencing remains deferred under the agreed scope and is still called out in the PR description; this change addresses the missing test, not the ownership protocol.

[VaguelySerious]

AI Review: Note

Listener connection and LISTEN failures are silently swallowed. Falling back is correct, but operators cannot distinguish healthy notification delivery from continuous one-second polling. Consider logging or telemetry for disconnects, reconnects, and fallback operation.

[shalabhc]

Addressed in e3b9a6b. Invocation notification degradation and restoration now emit bounded state-transition diagnostics to stderr. Repeated failed reconnects within an outage do not repeat the warning. Diagnostics use fixed failure categories and omit raw connection errors/options and payloads. Tests verify the degradation/restoration pair, suppression across repeated failures, and omission of sensitive error text.

[VaguelySerious]

AI Review: Note

There is no load or connection-footprint test for the extra dedicated connection per World, per-invocation Graphile wake, and repeated authoritative reads. A modest concurrency soak would help quantify the opt-in mode’s database cost and catch mailbox/index growth regressions.

[shalabhc]

Added a bounded real-Postgres/Graphile concurrency fixture in e3b9a6b: 72 inputs across 36 two-hook runs, in three parallel waves. It checks one shared producer LISTEN connection, the expected completed mailbox-row count, drained run executor queues, and zero listener connections after close; it also records caller result-read counts. This quantifies the test configuration and catches growth/lifecycle regressions, rather than claiming production throughput or a multi-host soak.