Backport #2472: perf(core): memoize step return value hydration across inline replays#2563
Merged
Merged
Conversation
…#2472) * perf(core): memoize step return value hydration across replays The inline replay loop re-executes the workflow body and re-consumes the full event log on every iteration. For each already-completed step, the step consumer re-decrypted and re-devalue-parsed the serialized result on every replay — O(N^2) decrypt+parse operations across a single invocation of a sequential N-step workflow. Add a per-run memoization cache, owned by the inline loop in runtime.ts (alongside cachedEvents) so it survives across replay iterations of the same run but never leaks across runs. It is threaded into runWorkflow and stored on the orchestrator context, and consulted in the step_completed path keyed by the persisted event id. This makes a completed step's hydrated result O(1) on subsequent replays, turning the aggregate cost into O(N). Determinism is preserved: the cache lookup happens inside the existing ctx.promiseQueue slot and still resolves via the same resolve(), so a cache hit occupies the identical position in the ordered delivery chain a re-hydrate would have — pendingDeliveries accounting, delivery barriers, and Promise.race/all replay are untouched. Identity safety: hydrateStepReturnValue returns a fresh object graph each call and each replay runs in a fresh VM, so sharing an object reference across replays could let one replay's mutation leak into the next. Only primitive results are memoized (immutable, reference-share == re-parse); non-primitives re-hydrate fresh every replay, exactly as before. Hook, wait, and abort hydration paths are intentionally left uncached. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * perf(core): bound memoized step-hydration cache by primitive size Address the review note that the per-run step hydration cache was never size-bounded: cached entries hold the decrypted/parsed plaintext of a primitive step result for the whole invocation, on top of the serialized bytes already retained in cachedEvents, so a long run returning large strings could roughly double peak retained memory for those results. Document the cache's memory characteristic (per-invocation, freed when the invocation ends, bounded by primitive-returning step count) and cap the only primitive types that can carry a large payload: string/bigint results longer than MAX_MEMOIZED_PRIMITIVE_LENGTH (4 KiB) fall through to the existing per-replay re-hydrate path instead of being memoized. Large payloads are cheap to re-hydrate relative to their footprint, so this caps the worst case at negligible cost. Other primitives are inherently small and always memoized. The cap only ever reduces what is cached, so deterministic replay is unaffected: oversized values take the already-correct re-hydrate path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Pranay Prakash <pranay.gp@gmail.com>
🦋 Changeset detectedLatest commit: 5d959da The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Contributor
Author
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests🌍 Community Worlds (99 failed)redis (19 failed):
turso (80 failed):
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
✅ 📋 Other
|
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
VaguelySerious
approved these changes
Jun 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated backport of #2472 to
stable(backport job run).AI recommendation: This is a self-contained performance optimization to
packages/core(memoizing step-return-value hydration during inline replay), and I verified that all modified files (step.ts,workflow.ts,runtime.ts,private.ts) and the exact hotspot it targets (hydrateStepReturnValueinside thepromiseQueueslot withpendingDeliveries) already exist onstable. The change is backward-compatible (new params/context field are optional, degrading to prior behavior) and doesn't depend on any main-only APIs, so it applies cleanly to the existing replay logic onstable.Merge conflicts were resolved by AI (opencode with
anthropic/claude-opus-4.8). Please review the conflict resolution carefully before merging.