You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/content/docs/agents/communication-channels/chat/autonomous-responses.mdx
+3-1Lines changed: 3 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -504,7 +504,9 @@ if (result.status === "aborted") {
504
504
505
505
</TypeScriptExample>
506
506
507
-
`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, and the signal is in memory only. If the Durable Object hibernates mid-turn and chat recovery is enabled, the recovered turn usually continues without the original signal; for pre-stream interruptions, recovery can instead retry the latest unanswered user message automatically. An abort fired after restart has no effect on the recovered turn.
507
+
`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, and the signal is in memory only. If the Durable Object hibernates mid-turn, durable recovery usually continues without the original signal. For pre-stream interruptions, recovery can retry the latest unanswered user message. An abort fired after restart has no effect on the recovered turn.
508
+
509
+
Persist cancellation intent when cancellation must survive a restart. Read that state in `onChatRecovery()` and return `{ continue: false }` to prevent another model call.
508
510
509
511
Use `cancelSubmission(submissionId)` for durable cancellation when work was accepted with `submitMessages()` or when cancellation must cross Worker and Durable Object RPC boundaries.
Copy file name to clipboardExpand all lines: src/content/docs/agents/communication-channels/chat/chat-agents.mdx
+21-26Lines changed: 21 additions & 26 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -399,7 +399,7 @@ if (result.status === "aborted") {
399
399
400
400
</TypeScriptExample>
401
401
402
-
`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, so construct the controller inside the Durable Object that calls `saveMessages()` or `continueLastTurn()`. The signal is in memory only; if the Durable Object hibernates mid-turn and `chatRecovery` is enabled, the recovered turn runs without the original signal.
402
+
`continueLastTurn()` accepts the same `options.signal` argument. `AbortSignal` objects cannot cross Durable Object RPC boundaries, so construct the controller inside the Durable Object that calls `saveMessages()` or `continueLastTurn()`. The signal is in memory only. If the Durable Object hibernates mid-turn, the recovered turn runs without the original signal. Persist cancellation intent when cancellation must survive a restart.
403
403
404
404
### `onChatResponse`
405
405
@@ -577,25 +577,13 @@ Use `abortRequest()` when you know the request ID. Use `abortAllRequests()` for
577
577
578
578
### Stream recovery
579
579
580
-
Automatic stream resumption (the `resume` option on `useAgentChat`) is **client reconnect recovery** — it resumes an active stream when a client disconnects and reconnects. It does not cover Durable Object eviction: if the Worker process or Durable Object is evicted while the model call is in flight, the stream itself is gone. `chatRecovery` handles that case.
580
+
Automatic stream resumption (the `resume` option on `useAgentChat`) is **client reconnect recovery** — it resumes an active stream when a client disconnects and reconnects. It does not cover Durable Object eviction. If the Worker process or Durable Object is evicted while the model call is in flight, the stream itself is gone. Durable chat recovery handles that case.
581
581
582
-
When a Durable Object is evicted mid-stream (code update, inactivity timeout, resource limit), the LLM connection is severed permanently and the in-memory streaming state is lost. `chatRecovery` wraps each chat turn in a [`runFiber()`](/agents/runtime/execution/durable-execution/), providing automatic `keepAlive` during streaming and a recovery hook on restart.
582
+
A mid-stream Durable Object eviction permanently severs the LLM connection. Durable recovery wraps every `AIChatAgent` and [`Think`](/agents/harnesses/think/)chat turn in a [`runFiber()`](/agents/runtime/execution/durable-execution/). The fiber provides automatic `keepAlive` during streaming and a recovery hook on restart.
583
583
584
-
<TypeScriptExample>
585
-
586
-
```ts
587
-
exportclassChatAgentextendsAIChatAgent {
588
-
override chatRecovery =true;
589
-
}
590
-
```
591
-
592
-
</TypeScriptExample>
584
+
The fiber row survives in SQLite after an eviction. On the next activation, the framework detects the interrupted fiber. It reconstructs the partial response from buffered stream chunks and calls `onChatRecovery`.
593
585
594
-
`AIChatAgent` defaults `chatRecovery` to `false`, so existing chat agents only get client reconnect and resumable-stream behavior unless they opt in. [`Think`](/agents/harnesses/think/) defaults it to `true`.
595
-
596
-
When enabled, every `onChatMessage` call runs inside a fiber. If the agent is evicted mid-stream, the fiber row survives in SQLite. On the next activation, the framework detects the interrupted fiber, reconstructs the partial response from buffered stream chunks, and calls `onChatRecovery`.
597
-
598
-
`chatRecovery` can also be set to a configuration object to bound recovery and customize the terminal experience when recovery cannot succeed:
586
+
Durable recovery is always on. Use `chatRecovery` only to tune recovery budgets and terminal behavior:
599
587
600
588
<TypeScriptExample>
601
589
@@ -608,8 +596,8 @@ export class ChatAgent extends AIChatAgent {
608
596
// Primary stuck-turn bound. Resets on every progress-bearing attempt, so a
609
597
// turn that keeps producing content survives unbounded interruption.
610
598
noProgressTimeoutMs: 5*60*1000,
611
-
// Runaway-loop guard. Defaults to Infinity (no cap). Set a finite value to
612
-
//seal a turn that keeps emitting content but never converges.
599
+
// Runaway-loop guard. Defaults to 1,000. Set a higher value for a long
600
+
//agentic turn, or Infinity to remove the cap.
613
601
maxRecoveryWork: 200,
614
602
// Caller policy consulted from the second recovery attempt onward. Return
615
603
// false to stop recovery. This is where you enforce a token/cost budget.
@@ -636,7 +624,8 @@ The `chatRecovery` object accepts the following configuration options:
636
624
|`stableTimeoutMs`|`10_000`| How long a recovery attempt waits for the isolate to reach stable state before rescheduling. |
637
625
|`terminalMessage`| generic message | The message shown to the user when recovery is given up on. |
638
626
|`noProgressTimeoutMs`|`300_000` (5 min) | Primary stuck-turn bound: how long an incident may go without forward progress before it is sealed (`no_progress_timeout`). **Resets on every progress-bearing attempt**, so a turn that keeps producing content survives unbounded interruption. |
639
-
|`maxRecoveryWork`|`Infinity`| Runaway-loop guard. Maximum produced content/tool units since the incident began before a still-progressing turn is sealed. Defaults to no cap. |
627
+
|`maxRecoveryWork`|`1,000`| Runaway-loop guard. Maximum produced content/tool units since the incident began before a still-progressing turn is sealed. Set a higher value or `Infinity` for a long agentic turn. |
628
+
|`maxOomRetries`|`3`| Retry budget for Durable Object memory-limit resets. Set `0` to stop after the first memory-limit reset. |
640
629
|`shouldKeepRecovering`| — | Caller policy consulted from the second recovery attempt onward. Return `false` to stop recovery. Use it to enforce a token or cost budget. `ctx.work` is a coarse segment count, not tokens, so track real spend yourself. |
641
630
|`onExhausted`| — | Called once when recovery is given up on, before the terminal message is delivered. Inspect `ctx.reason` for why. |
642
631
@@ -653,16 +642,17 @@ The `chatRecovery` object accepts the following configuration options:
653
642
|`work`|`number`| Coarse, monotonic count of content/tool segments produced since the incident opened (not tokens). |
654
643
|`ageMs`|`number`| Wall-clock ms since the incident's first interruption. |
655
644
656
-
A progressing turn is never terminated by the framework on its own — it survives unbounded interruption (for example a dense deploy window) as long as it keeps making forward progress. Recovery is sealed only by one of these `ctx.reason` values:
645
+
A progressing turn survives repeated interruptions as long as it stays within the `maxRecoveryWork` limit. Recovery is sealed by one of these `ctx.reason` values:
657
646
658
647
-`no_progress_timeout` — no forward progress within the no-progress window (a stuck turn).
659
648
-`max_attempts_exceeded` — the attempt cap was spent on a tight no-progress alarm loop.
660
649
-`work_budget_exceeded` — the turn kept producing content but exceeded `maxRecoveryWork` (a runaway loop).
661
650
-`recovery_aborted` — your `shouldKeepRecovering` hook returned `false`.
651
+
-`out_of_memory` — recovery exceeded the memory-limit retry budget.
662
652
-`stable_timeout` — recovery attempts kept timing out waiting for stable state until the budget drained (extreme churn).
663
653
664
654
:::tip
665
-
A finite `maxRecoveryWork`can seal a legitimately long turn. Set a cap well above what a healthy turn produces, or use `shouldKeepRecovering` with real token or cost accounting for a precise budget.
655
+
The `maxRecoveryWork`default prevents a progressing turn from running forever. Increase it for long agentic turns. Use `shouldKeepRecovering` with durable token or cost data for a precise budget.
Durable bookkeeping remains active when automatic continuation is not appropriate.
759
+
760
+
- Return `{ continue: false }` when another model call is unsafe.
761
+
- Persist cancellation intent and read it in `onChatRecovery()`.
762
+
- Record idempotency keys before external side effects.
763
+
- Use recovery budgets with durable spend data to limit cost.
764
+
768
765
#### `continueLastTurn`
769
766
770
767
Appends to the last assistant message by re-calling `onChatMessage` with the saved request body. The response is streamed as a continuation — appended to the existing assistant message, not a new one. No synthetic user message is created.
@@ -786,8 +783,6 @@ Use `this.stash()` inside `onChatMessage` to persist provider-specific data for
Copy file name to clipboardExpand all lines: src/content/docs/agents/concepts/agentic-patterns/long-running-agents.mdx
+2-4Lines changed: 2 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -540,7 +540,7 @@ For the full `subAgent()` API — typed RPC stubs, client routing, access contro
540
540
541
541
The patterns above handle the project manager's coordination work — scheduling, delegating, polling. But the project manager also uses an LLM directly: generating plans, summarizing progress, drafting status emails. Those LLM calls stream tokens over a connection that cannot be resumed if the agent is evicted mid-response.
542
542
543
-
For chat-oriented agents built on `AIChatAgent`, this is an even sharper problem — the user is watching the response stream in real time and sees it stop mid-sentence. `chatRecovery`wraps each chat turn in a `runFiber`, providing automatic `keepAlive` during streaming and a recovery hook when the agent restarts:
543
+
For chat-oriented agents built on `AIChatAgent` or `Think`, this is an even sharper problem — the user watches the response stream in real time and sees it stop mid-sentence. Durable recovery wraps every chat turn in a `runFiber`. This provides automatic `keepAlive` during streaming and a recovery hook when the agent restarts:
544
544
545
545
```ts
546
546
import { AIChatAgent } from"@cloudflare/ai-chat";
@@ -550,8 +550,6 @@ import type {
550
550
} from"@cloudflare/ai-chat";
551
551
552
552
classProjectChatextendsAIChatAgent<Env> {
553
-
override chatRecovery =true;
554
-
555
553
overrideasync onChatRecovery(
556
554
ctx:ChatRecoveryContext,
557
555
):Promise<ChatRecoveryOptions> {
@@ -575,7 +573,7 @@ The right recovery strategy depends on the LLM provider:
575
573
576
574
Use `ctx.createdAt` to suppress stale recoveries. For example, if a recovered chat turn is older than a few minutes, you may persist the partial answer but skip automatic continuation to avoid surprising the user with an old response.
577
575
578
-
[`Think`](/agents/harnesses/think/)enables `chatRecovery` by default. The default path persists partial output and auto-continues or retries the turn when safe, so many apps do not need a custom hook. Override `onChatRecovery` when a provider has a better recovery strategy, or configure `chatRecovery = { maxAttempts, terminalMessage, onExhausted }` to tune the terminal user experience.
576
+
`AIChatAgent` and [`Think`](/agents/harnesses/think/)always use durable recovery. The default path persists partial output and continues or retries the turn when safe. Override `onChatRecovery` when a provider has a better recovery strategy. Configure `chatRecovery = { maxAttempts, terminalMessage, onExhausted }` to tune the terminal experience.
579
577
580
578
If the agent is interrupted before any assistant stream chunks are written, there is no partial assistant message to continue. When the latest persisted message is still the unanswered user message from that turn, chat recovery retries the turn automatically unless `onChatRecovery` returns `{ continue: false }`.
Copy file name to clipboardExpand all lines: src/content/docs/agents/harnesses/think/client-tools.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -116,7 +116,7 @@ When a turn produces several client tool calls at once, Think waits for **all**
116
116
117
117
## Survive restarts while waiting for a human
118
118
119
-
A Durable Object can be evicted at any time, including while a turn is paused on an approval prompt or a client-side tool call. Because `Think`enables [`chatRecovery`](/agents/harnesses/think/recovery/)by default, the SDK treats such a turn as waiting on the human, not stuck. It parks the turn instead of failing it, and the user's eventual approval or tool result resumes the conversation.
119
+
A Durable Object can be evicted at any time, including while a turn is paused on an approval prompt or a client-side tool call. [`Think`durable recovery](/agents/harnesses/think/recovery/)is always on. The SDK treats such a turn as waiting on the human, not stuck. It parks the turn instead of failing it. The user's eventual approval or tool result resumes the conversation.
120
120
121
121
For which interactions are exempt from recovery budgets, refer to [Turns waiting on a human are not sealed](/agents/communication-channels/chat/chat-agents/#turns-waiting-on-a-human-are-not-sealed).
Copy file name to clipboardExpand all lines: src/content/docs/agents/harnesses/think/configuration.mdx
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,8 +31,8 @@ Think is configured by overriding methods and properties on your `Think` subclas
31
31
|`messageConcurrency`|`"queue"`| How overlapping submits behave — refer to [Client tools](/agents/harnesses/think/client-tools/#message-concurrency)|
32
32
|`includeMcpTools`|`true`| Convert connected MCP tools to AI SDK tools and add them to model turns. Refer to [MCP tools](/agents/harnesses/think/tools/#mcp-tools)|
33
33
|`waitForMcpConnections`|`false`| Wait for MCP servers before inference |
34
-
|`chatRecovery`|`true`| Wrap WebSocket, sub-agent, programmatic, and continuation turns in `runFiber` for durable execution. Set to a configuration object with `maxAttempts`, `stableTimeoutMs`, `terminalMessage`, and `onExhausted` to tune bounded recovery|
35
-
|`chatStreamStallTimeoutMs`|`0` (off) | Opt-in inactivity watchdog: abort a turn whose model stream produces no chunk for this long (measures the gap between chunks, including tool execution). With `chatRecovery` on, a stall routes into bounded recovery |
34
+
|`chatRecovery`|Always on| Durable recovery configuration. Refer to [Durable recovery](/agents/harnesses/think/recovery/) for all options and defaults |
35
+
|`chatStreamStallTimeoutMs`|`0` (off) | Opt-in inactivity watchdog: abort a turn whose model stream produces no chunk for this long (measures the gap between chunks, including tool execution). A stall routes into bounded recovery|
36
36
|`contextOverflow`|`undefined`| Opt-in mid-turn context-overflow handling with `reactive`, `maxRetries`, and `proactive` options. Requires `classifyChatError` plus a session compaction function — refer to [Context-window overflow recovery](/agents/harnesses/think/recovery/#context-window-overflow-recovery)|
37
37
38
38
For `chatRecovery` and `chatStreamStallTimeoutMs` behavior, refer to [Durable recovery](/agents/harnesses/think/recovery/).
Copy file name to clipboardExpand all lines: src/content/docs/agents/harnesses/think/index.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -247,7 +247,7 @@ Key behaviors:
247
247
248
248
-**Blocking modes cannot nest.** Calling `wait`/`stream`/`continuation` (or the equivalent shortcut) from _inside_ an active turn — for example, from a tool's `execute` — throws, because it would deadlock the turn queue. From inside a turn, use `runTurn({ mode: "submit" })` (durable, runs after the current turn frees the queue) or [`addMessages()`](#add-messages-without-a-turn) (transcript only, no inference).
249
249
-**`submit` is idempotent.** Pass `submissionId` and/or `idempotencyKey`; re-submitting a known key returns the existing record with `accepted: false` instead of starting a second turn. See [Programmatic submissions](/agents/harnesses/think/programmatic-submissions/).
250
-
-**Recovery-safe.**When `chatRecovery` is enabled, the `wait`, `stream`, and drained `submit` paths all run inference inside a recovery fiber, so an interrupted turn resumes after eviction.
250
+
-**Recovery-safe.**The `wait`, `stream`, and drained `submit` paths run inference inside a recovery fiber, so an interrupted turn resumes after eviction.
251
251
252
252
`runTurn` is exported alongside its option and result types: `RunTurnOptions`, `RunTurnWait`, `RunTurnSubmit`, `RunTurnStream`, `TurnInputMessages`, and `TurnResult`.
0 commit comments