feat: configurable client-side message queue for useChat#900
Open
AlemTuzlak wants to merge 23 commits into
Open
feat: configurable client-side message queue for useChat#900AlemTuzlak wants to merge 23 commits into
AlemTuzlak wants to merge 23 commits into
Conversation
…unsubscribe Wires the queue drain into streamResponse's settle path so queued sends actually go out (fifo one-at-a-time, or batch merged with newlines), and flushes any pending queue on stop()/clear()/unsubscribe() so callers don't get a surprise send after tearing down. Also surfaces the queue on the devtools snapshot.
A RUN_ERROR or thrown non-abort error left streamCompletedSuccessfully false, so the finally block's queue handling (guarded by that flag) was skipped entirely. Queued messages were stranded until a later direct sendMessage sent first and drained the stale queued item afterward, inverting message order. Flush (not drain) the queue on the non-success settle path, matching stop()'s existing behavior.
Mirrors the ai-react implementation (17bb417): ChatClient's queue-by-default behavior is now wired through the Preact useChat hook via onQueueChange, with cancelQueued and per-send whenBusy passthrough. Updates the two tests that previously asserted a mid-stream second sendMessage was dropped to instead assert both messages land in order.
…react to changeset
Contributor
🚀 Changeset Version Preview9 package(s) bumped directly, 36 bumped as dependents. 🟥 Major bumps
🟨 Minor bumps
🟩 Patch bumps
|
|
View your CI Pipeline Execution ↗ for commit 7401f07
☁️ Nx Cloud last updated this comment at |
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (8)
✅ Files skipped from review due to trivial changes (2)
📝 WalkthroughWalkthroughThis PR adds default queueing for ChangesMessage Queueing Feature
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant useChat
participant ChatClient
participant queue
User->>useChat: sendMessage(content, { whenBusy })
useChat->>ChatClient: sendMessage(content, whenBusy)
ChatClient->>ChatClient: decideWhenBusy(isStreaming)
alt enqueue
ChatClient->>queue: add queued message
ChatClient->>useChat: onQueueChange(queue)
else interrupt
ChatClient->>ChatClient: stop in-flight stream
ChatClient->>ChatClient: send immediately
end
ChatClient-->>useChat: stream settles
ChatClient->>queue: drainQueue() or flushQueue()
ChatClient->>useChat: onQueueChange(queue)
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
@tanstack/ai
@tanstack/ai-acp
@tanstack/ai-angular
@tanstack/ai-anthropic
@tanstack/ai-bedrock
@tanstack/ai-claude-code
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-codex
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-grok-build
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-mcp
@tanstack/ai-mistral
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-opencode
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-sandbox
@tanstack/ai-sandbox-cloudflare
@tanstack/ai-sandbox-daytona
@tanstack/ai-sandbox-docker
@tanstack/ai-sandbox-local-process
@tanstack/ai-sandbox-sprites
@tanstack/ai-sandbox-vercel
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
The framework packages re-exported QueuedMessage/WhenBusy/QueueConfig/ QueueStrategy/QueueOption from their internal types module but never forwarded them through the public index entry, so importing them from e.g. @tanstack/ai-react failed at the package boundary.
Side-by-side panels (queue/fifo, interrupt, queue/batch) driven by a shared composer, showcasing how each queue strategy handles a message sent while a stream is in flight, with live queue rendering + cancel.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (1)
packages/ai-solid/src/use-chat.ts (1)
141-144: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winSame queue-sync gap as the React adapter.
options.queueis only captured once when the client is created; there's nocreateEffectre-applying it viaclient().updateOptions(...)the waybody/forwardedProps/contextare synced at Line 152-165. See the parallel comment onpackages/ai-react/src/use-chat.ts(Line 162-168) — same underlying question of whetherqueueis meant to be static-per-session or live-updatable.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-solid/src/use-chat.ts` around lines 141 - 144, The chat queue is only read once when the client is created in useChat, so later changes to options.queue are never synced into the live client. Add a createEffect alongside the existing body/forwardedProps/context syncing that calls client().updateOptions(...) whenever options.queue changes, using the same pattern as the other reactive option updates in useChat so the queue stays in sync if it is meant to be live-updatable.
🧹 Nitpick comments (3)
packages/ai-preact/src/use-chat.ts (1)
157-163: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value
queueconfig isn't kept in sync after mount.Unlike
body/forwardedProps/context(synced via theupdateOptionseffect below),queueis only read once at client construction (useMemokeyed on[clientId]). If a consumer changesoptions.queue(e.g. togglingwhenBusy) after mount without changingid, the change is silently ignored. This mirrors the existing behavior for other one-time options liketools/persistence, so it's consistent with current design, but worth confirming it's intentional for a config surface consumers may want to adjust dynamically (e.g., per-messagewhenBusyoverrides already exist for that case).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-preact/src/use-chat.ts` around lines 157 - 163, The `queue` option in `useChat` is only applied when the client is created in the `useMemo` block, so later changes to `options.queue` are ignored after mount. Update the `updateOptions` path in `use-chat.ts` (the effect that already keeps `body`, `forwardedProps`, and `context` in sync) to also propagate `queue` when `optionsRef.current.queue` changes, using the same `activeClientRef.current === instance` guard as `onQueueChange`. If `queue` is meant to stay one-time like `tools`/`persistence`, make that behavior explicit in the option handling and docs.packages/ai-client/src/chat-client.ts (1)
803-846: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDocument the new
sendOptionsparameter.The extensive JSDoc/example block for
sendMessagewasn't updated to mention the new third parameter (sendOptions.whenBusy), even though it's a public API addition central to this PR.📝 Suggested doc addition
* `@param` body - Optional body parameters to merge with the client's base body for this request. * Uses shallow merge with per-message body taking priority. + * `@param` sendOptions - Optional per-call overrides, e.g. `{ whenBusy: 'interrupt' }` to + * override the configured queue policy for this one send. * * `@example`🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-client/src/chat-client.ts` around lines 803 - 846, The public JSDoc for sendMessage is missing the new third parameter, so update the documentation block for sendMessage to describe sendOptions and its whenBusy option alongside content and body. Add a brief `@param` entry and, if helpful, a small example showing how sendOptions.whenBusy is used, so the docs match the new signature in chat-client.ts.packages/ai-client/tests/chat-client-queue.test.ts (1)
71-328: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd coverage for
interrupt, per-callwhenBusyoverride, andmaxSize/onOverflow.This suite covers default queueing, drop, FIFO/batch drain, and flush-on-stop/error well, but three documented behaviors have zero test coverage:
whenBusy: 'interrupt'(abort + send now)- per-call
sendOptions.whenBusyoverride onsendMessagemaxSizewithonOverflow: 'reject'vs'drop-oldest'Adding tests for the overflow behavior in particular would surface the
maxSize: 0boundary issue flagged inchat-client.ts.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-client/tests/chat-client-queue.test.ts` around lines 71 - 328, Add tests in ChatClient message queue coverage for the missing documented behaviors: verify `whenBusy: 'interrupt'` aborts the in-flight send and allows the new message to send immediately, verify `sendMessage` honors a per-call `sendOptions.whenBusy` override over the client default, and verify `maxSize` with `onOverflow: 'reject'` and `'drop-oldest'` behaves correctly. Use the existing `ChatClient`, `sendMessage`, `getQueue`, `getMessages`, and the held connection helpers as reference points, and add a boundary case for `maxSize: 0` to cover the overflow path mentioned in `chat-client.ts`.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/ai-client/src/chat-client.ts`:
- Around line 905-923: The enqueueMessage queue limit check currently lets one
item through when queueConfig.maxSize is 0 and onOverflow is not 'reject',
because it shifts an empty queue and then still pushes a new message. Update
enqueueMessage to treat maxSize: 0 as a hard cap in the same way as other
full-queue cases, using the existing messageQueue, queueConfig, and onOverflow
logic so no item is enqueued when the limit is zero.
- Around line 882-923: The queued item id is generated twice, so the
`pending.id` seen by `QueueStrategy` does not match the actual entry stored by
`enqueueMessage`. Generate one queued id in `decideWhenBusy`, pass it through to
`enqueueMessage`, and use that same id when building the `pending` context and
when pushing the message so `getQueue()`, `onQueueChange`, and
`cancelQueued(id)` all refer to the same item.
In `@packages/ai-svelte/src/types.ts`:
- Around line 149-152: The `queue` property in the chat state interface is
missing `readonly`, unlike the other reactive getters and the Svelte/Vue
equivalents. Update the `queue` declaration in `types.ts` to be readonly
alongside `messages`, `isLoading`, `error`, `status`, `isSubscribed`,
`connectionStatus`, and `sessionGenerating`, so consumers know it is getter-only
and cannot be assigned to directly.
---
Duplicate comments:
In `@packages/ai-solid/src/use-chat.ts`:
- Around line 141-144: The chat queue is only read once when the client is
created in useChat, so later changes to options.queue are never synced into the
live client. Add a createEffect alongside the existing
body/forwardedProps/context syncing that calls client().updateOptions(...)
whenever options.queue changes, using the same pattern as the other reactive
option updates in useChat so the queue stays in sync if it is meant to be
live-updatable.
---
Nitpick comments:
In `@packages/ai-client/src/chat-client.ts`:
- Around line 803-846: The public JSDoc for sendMessage is missing the new third
parameter, so update the documentation block for sendMessage to describe
sendOptions and its whenBusy option alongside content and body. Add a brief
`@param` entry and, if helpful, a small example showing how sendOptions.whenBusy
is used, so the docs match the new signature in chat-client.ts.
In `@packages/ai-client/tests/chat-client-queue.test.ts`:
- Around line 71-328: Add tests in ChatClient message queue coverage for the
missing documented behaviors: verify `whenBusy: 'interrupt'` aborts the
in-flight send and allows the new message to send immediately, verify
`sendMessage` honors a per-call `sendOptions.whenBusy` override over the client
default, and verify `maxSize` with `onOverflow: 'reject'` and `'drop-oldest'`
behaves correctly. Use the existing `ChatClient`, `sendMessage`, `getQueue`,
`getMessages`, and the held connection helpers as reference points, and add a
boundary case for `maxSize: 0` to cover the overflow path mentioned in
`chat-client.ts`.
In `@packages/ai-preact/src/use-chat.ts`:
- Around line 157-163: The `queue` option in `useChat` is only applied when the
client is created in the `useMemo` block, so later changes to `options.queue`
are ignored after mount. Update the `updateOptions` path in `use-chat.ts` (the
effect that already keeps `body`, `forwardedProps`, and `context` in sync) to
also propagate `queue` when `optionsRef.current.queue` changes, using the same
`activeClientRef.current === instance` guard as `onQueueChange`. If `queue` is
meant to stay one-time like `tools`/`persistence`, make that behavior explicit
in the option handling and docs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 384b81cf-49eb-43ca-a54f-13b3dc63cefd
📒 Files selected for processing (28)
.changeset/message-queue.mddocs/chat/streaming.mddocs/config.jsonpackages/ai-client/src/chat-client.tspackages/ai-client/src/devtools.tspackages/ai-client/src/index.tspackages/ai-client/src/types.tspackages/ai-client/tests/chat-client-queue.test.tspackages/ai-client/tests/chat-client.test.tspackages/ai-preact/src/types.tspackages/ai-preact/src/use-chat.tspackages/ai-preact/tests/use-chat.test.tspackages/ai-react/src/types.tspackages/ai-react/src/use-chat.tspackages/ai-react/tests/use-chat.test.tspackages/ai-solid/src/types.tspackages/ai-solid/src/use-chat.tspackages/ai-solid/tests/use-chat.test.tspackages/ai-svelte/src/create-chat.svelte.tspackages/ai-svelte/src/types.tspackages/ai-vue/src/types.tspackages/ai-vue/src/use-chat.tspackages/ai-vue/tests/use-chat.test.tspackages/ai/skills/ai-core/chat-experience/SKILL.mdtesting/e2e/fixtures/queue/basic.jsontesting/e2e/src/components/ChatUI.tsxtesting/e2e/src/routes/$provider/$feature.tsxtesting/e2e/tests/queue.spec.ts
Comment on lines
+882
to
+923
| private decideWhenBusy( | ||
| content: string | MultimodalContent, | ||
| sendOptions?: SendMessageOptions, | ||
| ): 'drop' | 'enqueue' | 'interrupt' { | ||
| if (sendOptions?.whenBusy) { | ||
| return sendOptions.whenBusy === 'queue' ? 'enqueue' : sendOptions.whenBusy | ||
| } | ||
| const { strategy, whenBusy } = this.queueConfig | ||
| if (strategy) { | ||
| const { action } = strategy({ | ||
| pending: { | ||
| id: this.generateUniqueId('queued'), | ||
| content, | ||
| createdAt: Date.now(), | ||
| }, | ||
| isStreaming: true, | ||
| queued: this.getQueue(), | ||
| }) | ||
| return action === 'send' ? 'enqueue' : action | ||
| } | ||
| return whenBusy === 'queue' ? 'enqueue' : whenBusy | ||
| } | ||
|
|
||
| private enqueueMessage( | ||
| content: string | MultimodalContent, | ||
| body?: Record<string, any>, | ||
| ): void { | ||
| const { maxSize, onOverflow } = this.queueConfig | ||
| if (maxSize !== undefined && this.messageQueue.length >= maxSize) { | ||
| if (onOverflow === 'reject') { | ||
| return | ||
| } | ||
| this.messageQueue.shift() // drop-oldest | ||
| } | ||
| this.messageQueue.push({ | ||
| id: this.generateUniqueId('queued'), | ||
| content, | ||
| createdAt: Date.now(), | ||
| ...(body !== undefined ? { body } : {}), | ||
| }) | ||
| this.emitQueueChange() | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
pending.id shown to a QueueStrategy never matches the actual enqueued item's id.
decideWhenBusy generates one id (Line 893) to build the pending context passed to a custom QueueStrategy, but if the strategy returns 'enqueue', enqueueMessage (Line 916-921) generates a different id for the actual queued entry. Any strategy that inspects pending.id (e.g. to later cancelQueued(id) the very message it just classified) will silently fail to match, since the id it saw is never the one stored/exposed via getQueue()/onQueueChange.
🐛 Proposed fix: thread a single id through decideWhenBusy → enqueueMessage
private decideWhenBusy(
content: string | MultimodalContent,
sendOptions?: SendMessageOptions,
- ): 'drop' | 'enqueue' | 'interrupt' {
+ ): { action: 'drop' | 'enqueue' | 'interrupt'; id: string } {
+ const id = this.generateUniqueId('queued')
if (sendOptions?.whenBusy) {
- return sendOptions.whenBusy === 'queue' ? 'enqueue' : sendOptions.whenBusy
+ return {
+ action: sendOptions.whenBusy === 'queue' ? 'enqueue' : sendOptions.whenBusy,
+ id,
+ }
}
const { strategy, whenBusy } = this.queueConfig
if (strategy) {
const { action } = strategy({
- pending: {
- id: this.generateUniqueId('queued'),
- content,
- createdAt: Date.now(),
- },
+ pending: { id, content, createdAt: Date.now() },
isStreaming: true,
queued: this.getQueue(),
})
- return action === 'send' ? 'enqueue' : action
+ return { action: action === 'send' ? 'enqueue' : action, id }
}
- return whenBusy === 'queue' ? 'enqueue' : whenBusy
+ return { action: whenBusy === 'queue' ? 'enqueue' : whenBusy, id }
}
private enqueueMessage(
content: string | MultimodalContent,
body?: Record<string, any>,
+ id?: string,
): void {
const { maxSize, onOverflow } = this.queueConfig
if (maxSize !== undefined && this.messageQueue.length >= maxSize) {
if (onOverflow === 'reject') {
return
}
this.messageQueue.shift() // drop-oldest
}
this.messageQueue.push({
- id: this.generateUniqueId('queued'),
+ id: id ?? this.generateUniqueId('queued'),
content,
createdAt: Date.now(),
...(body !== undefined ? { body } : {}),
})
this.emitQueueChange()
}And update the call site:
if (this.isLoading) {
- const action = this.decideWhenBusy(content, sendOptions)
+ const { action, id } = this.decideWhenBusy(content, sendOptions)
if (action === 'drop') {
return
}
if (action === 'enqueue') {
- this.enqueueMessage(content, body)
+ this.enqueueMessage(content, body, id)
return
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| private decideWhenBusy( | |
| content: string | MultimodalContent, | |
| sendOptions?: SendMessageOptions, | |
| ): 'drop' | 'enqueue' | 'interrupt' { | |
| if (sendOptions?.whenBusy) { | |
| return sendOptions.whenBusy === 'queue' ? 'enqueue' : sendOptions.whenBusy | |
| } | |
| const { strategy, whenBusy } = this.queueConfig | |
| if (strategy) { | |
| const { action } = strategy({ | |
| pending: { | |
| id: this.generateUniqueId('queued'), | |
| content, | |
| createdAt: Date.now(), | |
| }, | |
| isStreaming: true, | |
| queued: this.getQueue(), | |
| }) | |
| return action === 'send' ? 'enqueue' : action | |
| } | |
| return whenBusy === 'queue' ? 'enqueue' : whenBusy | |
| } | |
| private enqueueMessage( | |
| content: string | MultimodalContent, | |
| body?: Record<string, any>, | |
| ): void { | |
| const { maxSize, onOverflow } = this.queueConfig | |
| if (maxSize !== undefined && this.messageQueue.length >= maxSize) { | |
| if (onOverflow === 'reject') { | |
| return | |
| } | |
| this.messageQueue.shift() // drop-oldest | |
| } | |
| this.messageQueue.push({ | |
| id: this.generateUniqueId('queued'), | |
| content, | |
| createdAt: Date.now(), | |
| ...(body !== undefined ? { body } : {}), | |
| }) | |
| this.emitQueueChange() | |
| } | |
| private decideWhenBusy( | |
| content: string | MultimodalContent, | |
| sendOptions?: SendMessageOptions, | |
| ): { action: 'drop' | 'enqueue' | 'interrupt'; id: string } { | |
| const id = this.generateUniqueId('queued') | |
| if (sendOptions?.whenBusy) { | |
| return { | |
| action: sendOptions.whenBusy === 'queue' ? 'enqueue' : sendOptions.whenBusy, | |
| id, | |
| } | |
| } | |
| const { strategy, whenBusy } = this.queueConfig | |
| if (strategy) { | |
| const { action } = strategy({ | |
| pending: { id, content, createdAt: Date.now() }, | |
| isStreaming: true, | |
| queued: this.getQueue(), | |
| }) | |
| return { action: action === 'send' ? 'enqueue' : action, id } | |
| } | |
| return { action: whenBusy === 'queue' ? 'enqueue' : whenBusy, id } | |
| } | |
| private enqueueMessage( | |
| content: string | MultimodalContent, | |
| body?: Record<string, any>, | |
| id?: string, | |
| ): void { | |
| const { maxSize, onOverflow } = this.queueConfig | |
| if (maxSize !== undefined && this.messageQueue.length >= maxSize) { | |
| if (onOverflow === 'reject') { | |
| return | |
| } | |
| this.messageQueue.shift() // drop-oldest | |
| } | |
| this.messageQueue.push({ | |
| id: id ?? this.generateUniqueId('queued'), | |
| content, | |
| createdAt: Date.now(), | |
| ...(body !== undefined ? { body } : {}), | |
| }) | |
| this.emitQueueChange() | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-client/src/chat-client.ts` around lines 882 - 923, The queued
item id is generated twice, so the `pending.id` seen by `QueueStrategy` does not
match the actual entry stored by `enqueueMessage`. Generate one queued id in
`decideWhenBusy`, pass it through to `enqueueMessage`, and use that same id when
building the `pending` context and when pushing the message so `getQueue()`,
`onQueueChange`, and `cancelQueued(id)` all refer to the same item.
Comment on lines
+905
to
+923
| private enqueueMessage( | ||
| content: string | MultimodalContent, | ||
| body?: Record<string, any>, | ||
| ): void { | ||
| const { maxSize, onOverflow } = this.queueConfig | ||
| if (maxSize !== undefined && this.messageQueue.length >= maxSize) { | ||
| if (onOverflow === 'reject') { | ||
| return | ||
| } | ||
| this.messageQueue.shift() // drop-oldest | ||
| } | ||
| this.messageQueue.push({ | ||
| id: this.generateUniqueId('queued'), | ||
| content, | ||
| createdAt: Date.now(), | ||
| ...(body !== undefined ? { body } : {}), | ||
| }) | ||
| this.emitQueueChange() | ||
| } |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
maxSize: 0 with onOverflow: 'drop-oldest' still allows one item into the queue.
this.messageQueue.length >= maxSize is 0 >= 0 (true) on the very first enqueue, so it shifts (no-op on an empty queue) and then pushes anyway — leaving queue length 1, violating the documented "Max queued items" contract for the maxSize: 0 edge case.
🐛 Proposed fix
if (maxSize !== undefined && this.messageQueue.length >= maxSize) {
if (onOverflow === 'reject') {
return
}
+ if (maxSize === 0) {
+ return
+ }
this.messageQueue.shift() // drop-oldest
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| private enqueueMessage( | |
| content: string | MultimodalContent, | |
| body?: Record<string, any>, | |
| ): void { | |
| const { maxSize, onOverflow } = this.queueConfig | |
| if (maxSize !== undefined && this.messageQueue.length >= maxSize) { | |
| if (onOverflow === 'reject') { | |
| return | |
| } | |
| this.messageQueue.shift() // drop-oldest | |
| } | |
| this.messageQueue.push({ | |
| id: this.generateUniqueId('queued'), | |
| content, | |
| createdAt: Date.now(), | |
| ...(body !== undefined ? { body } : {}), | |
| }) | |
| this.emitQueueChange() | |
| } | |
| private enqueueMessage( | |
| content: string | MultimodalContent, | |
| body?: Record<string, any>, | |
| ): void { | |
| const { maxSize, onOverflow } = this.queueConfig | |
| if (maxSize !== undefined && this.messageQueue.length >= maxSize) { | |
| if (onOverflow === 'reject') { | |
| return | |
| } | |
| if (maxSize === 0) { | |
| return | |
| } | |
| this.messageQueue.shift() // drop-oldest | |
| } | |
| this.messageQueue.push({ | |
| id: this.generateUniqueId('queued'), | |
| content, | |
| createdAt: Date.now(), | |
| ...(body !== undefined ? { body } : {}), | |
| }) | |
| this.emitQueueChange() | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-client/src/chat-client.ts` around lines 905 - 923, The
enqueueMessage queue limit check currently lets one item through when
queueConfig.maxSize is 0 and onOverflow is not 'reject', because it shifts an
empty queue and then still pushes a new message. Update enqueueMessage to treat
maxSize: 0 as a hard cap in the same way as other full-queue cases, using the
existing messageQueue, queueConfig, and onOverflow logic so no item is enqueued
when the limit is zero.
Comment on lines
+149
to
+152
| /** | ||
| * Pending messages queued while a stream is in flight. | ||
| */ | ||
| queue: Array<QueuedMessage> |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Mark queue as readonly for consistency.
Every other reactive getter in this interface (messages, isLoading, error, status, isSubscribed, connectionStatus, sessionGenerating) is declared readonly, and the Svelte implementation only exposes queue via a getter (no setter). The Vue equivalent is also wrapped in Readonly<...>. Missing readonly here lets consumers believe chat.queue = [...] is a valid assignment.
♻️ Proposed fix
/**
* Pending messages queued while a stream is in flight.
*/
- queue: Array<QueuedMessage>
+ readonly queue: Array<QueuedMessage>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| /** | |
| * Pending messages queued while a stream is in flight. | |
| */ | |
| queue: Array<QueuedMessage> | |
| /** | |
| * Pending messages queued while a stream is in flight. | |
| */ | |
| readonly queue: Array<QueuedMessage> |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/ai-svelte/src/types.ts` around lines 149 - 152, The `queue` property
in the chat state interface is missing `readonly`, unlike the other reactive
getters and the Svelte/Vue equivalents. Update the `queue` declaration in
`types.ts` to be readonly alongside `messages`, `isLoading`, `error`, `status`,
`isSubscribed`, `connectionStatus`, and `sessionGenerating`, so consumers know
it is getter-only and cannot be assigned to directly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Today
sendMessagesilently drops any message sent while a stream is in flight. This adds a first-class, observable, configurable message queue: messages sent mid-stream are queued by default, rendered separately, auto-sent when the stream settles, and cancellable before they go out.Behavior
whenBusy—queue(hold + auto-send),drop(ignore, the old behavior),interrupt(abort current stream, send now).drain—fifo(one at a time, in order) orbatch(merge all queued into one send: strings joined by\n, multimodal content concatenated).maxSize/onOverflow— cap the queue;rejectthe incoming send or evict the oldest.WhenBusystring shorthand or aQueueStrategyfunction for full control.stop()/clear()/unsubscribe()and on stream error (never strands or mis-orders queued sends).messages, never sent to the wire until drained.Sends while streaming are now queued by default (previously dropped). Opt back into the old behavior with
queue: 'drop'. Flagged as a minor bump in the changeset.Surface
@tanstack/ai-client— core:queueoption,QueuedMessage,getQueue(),cancelQueued(),onQueueChange, FIFO/batch drain, lifecycle + error flush, devtools snapshot.@tanstack/ai-react/-solid/-vue/-svelte/-preact— exposequeue+cancelQueued, per-send{ whenBusy }override;ai-vue-uiforwards automatically.Tests & docs
ai-client(eachwhenBusymode, FIFO order, batch merge incl. multimodal,maxSize/overflow, cancel, flush-on-stop/clear/error) + framework hook tests updated for queue-by-default.testing/e2e/tests/queue.spec.ts): send → queue-while-streaming → cancel one → assert only the uncancelled message is delivered, in order.docs/chat/streaming.md;chat-experienceagent skill updated.pnpm test:prgreen across all 50 projects.🤖 Generated with Claude Code
Summary by CodeRabbit
whenBusyoverrides.