google: 400 'Requests ending with a model turn are not supported' after barge-in on gemini-3.6-flash — retried in a loop, agent goes silent

[arsenieciprian]

Summary

When the user interrupts the agent (barge-in) in a cascaded AgentSession using the Google plugin (Vertex AI), the truncated assistant message is committed to the chat context after the user's transcript. The next LLM request is then sent with a history that ends with a model turn, which newer Gemini models (observed with gemini-3.6-flash) reject deterministically:

400 INVALID_ARGUMENT: "Requests ending with a model turn are not supported."

The plugin marks the error as retryable: true, so LLMStream.run retries the identical request in a loop (3 attempts in our runs) — every retry fails with the same 400, and the agent goes silent for ~25 seconds until the user happens to speak again, which appends a user turn and unblocks the conversation.

Older models (gemini-3.1-flash-lite, gemini-3.5-flash-lite) tolerate trailing model turns, so the bug only surfaces on newer models. Edit 2026-07-27: no longer accurate — gemini-3.5-flash-lite reproduces this in production too (84 errors / 21 lost turns / 7 calls in ~24h). See the update comment below. Google has documented this as intended API behavior for newer Gemini models (prefilled model turns are no longer supported) — see references below.

Environment

• @livekit/agents: 1.5.5
• @livekit/agents-plugin-google: 1.5.5 (LLM via Vertex AI, vertexai: true, location global)
• Model: gemini-3.6-flash (also expected on other current-gen models enforcing the constraint)
• Node.js v24.18.0, Linux (Rocky 10)
• Cascaded pipeline: Google Cloud STT → google.LLM → Google Cloud TTS, local VAD (inference.VAD), allowInterruptions: true
• Telephony (LiveKit SIP), Romanian language — irrelevant to the bug, included for completeness

Timeline from production logs (redacted)

The agent proposes an address; the user barges in with a confirmation while the agent is speaking; the agent's reply is truncated by the interruption:

14:07:22 [METRIC] llm ttft=849ms dur=1161ms
14:07:29 [USER]  Da, da, da, da, acolo.          <- user turn committed (barge-in)
14:07:29 [AGENT] Am găsit Liceul Teoretic Dumitru Tăuțan, de pe strada   <- truncated by interruption,
                                                    committed AFTER the user turn
14:07:29 {"llm":"google.LLM","attempt":1,"error":{"type":"APIConnectionError","message":"Google LLM: API error -
  {\"error\":{\"code\":400,\"message\":\"Requests ending with a model turn are not supported.\",
  \"status\":\"INVALID_ARGUMENT\"}}","retryable":true},"msg":"failed to generate LLM completion, retrying in 0.1ms"}
14:07:29 ... attempt=2, same 400 ...
14:07:32 ... attempt=3, same 400 ...
14:07:34 [METRIC] llm ttft=-1ms dur=5381ms        <- generation gave up
        (~25s of dead air on the phone call)
14:07:49 [USER]  Alo                              <- user speaks again -> a user turn is appended
14:07:50 [METRIC] llm ttft=961ms dur=1250ms       <- next request succeeds

Root cause

LLMStream.run in agents-plugin-google/src/llm.ts sends the chat history exactly as produced by chatCtx.toProviderFormat('google'), with no normalization of the final turn:

const [turns, extraData] = await this.chatCtx.toProviderFormat("google");
const contents = turns.map((turn) => ({ role: turn.role, parts: turn.parts }));

After a barge-in, the committed order in the chat context is [..., user: "Da, da...", model: "<truncated reply>"], so contents ends with a model turn. Newer Gemini models reject that request with a deterministic 400.

Two compounding problems:

1. No sanitization of the trailing turn for the Google provider, even though the Gemini API now requires the last content to be a user turn.
2. The 400 is treated as retryable — the identical request is retried and fails identically, turning a deterministic client-side formatting issue into a multi-second outage of the voice session.

Expected behavior

The plugin (or toProviderFormat('google')) should guarantee the request never ends with a model turn — e.g. by reordering a trailing assistant message before the preceding user message (chronologically faithful for the interruption case: the agent spoke the partial reply, then the user reacted), or by any equivalent normalization. Deterministic INVALID_ARGUMENT responses should also not be retried with an unchanged payload.

Repro steps

1. Cascaded AgentSession with google.LLM on gemini-3.6-flash (Vertex, any location), any STT/TTS, allowInterruptions: true.
2. Let the agent produce a reply of a few seconds.
3. Interrupt it mid-sentence with a short user utterance (so the truncated assistant message is committed after the user transcript).
4. The next generation fails with 400 INVALID_ARGUMENT: "Requests ending with a model turn are not supported." and is retried in a loop; the session produces no speech until a new user turn arrives.

References

• Google Gemini API: prefilled/trailing model turns are not supported on newer models (400 INVALID_ARGUMENT) — https://ai.google.dev/gemini-api/docs/troubleshooting
• Same constraint hit by other frameworks: .Net: Bug: Gemini does not accept 'model' / assistant role as a last message in ChatHistory when pass over to the next agent. microsoft/semantic-kernel#13262

[arsenieciprian]

Update: this also affects gemini-3.5-flash-lite — the note in the original report is no longer accurate

The report states that gemini-3.1-flash-lite and gemini-3.5-flash-lite tolerate trailing model turns. That is no longer true for gemini-3.5-flash-lite. After moving our production agents to it on 2026-07-26, the same deterministic 400 started appearing on live calls.

Figures since the switch (single telephony agent, ~24h, unchanged versions @livekit/agents 1.5.5 / @livekit/agents-plugin-google 1.5.5, Vertex global):

metric	value
total 400 … model turn errors	84
unrecoverable (retries exhausted → turn lost)	21
distinct calls affected	7
affected calls that ended without a completed booking	6

Two implications:

1. This is not limited to a next-gen model — it now hits a GA model that is a reasonable production default, so the blast radius is wider than the original report suggested.
2. Because the 400 is deterministic and the retry re-sends an identical payload, every occurrence costs a full retry cycle followed by dead air until the caller speaks again. On a phone call this reads as a dropped line.

The local workaround we are evaluating is reordering a trailing assistant message before the preceding user turn in llmNode, before the provider payload is built. Happy to test a patch if one lands.

[arsenieciprian]

Root cause found — it's a one-line bug in provider_format/google.ts, not a missing feature

The guard that should prevent this already exists and is enabled by default (injectDummyUserMessage = true). It just doesn't fire in the barge-in case, because it tests the loop state variable instead of the turns actually produced:

// agents/src/llm/provider_format/google.ts (1.5.5, dist line 77)
if (injectDummyUserMessage && currentRole !== "user") {
  turns.push({ role: "user", parts: [{ text: "." }] });
}

When the last item in the chat context is a user message with empty content — exactly what an interruption produces ("message":"", "msg":"playout completed with interrupt") — the loop sets currentRole = "user" but never appends the turn, because parts is empty:

if (currentRole !== null && parts.length > 0) {   // <- empty user turn is dropped here
  turns.push({ role: currentRole, parts });
}
if (injectDummyUserMessage && currentRole !== "user") {   // <- but currentRole is "user", so no injection

So turns ends with model, and Vertex rejects the request.

Minimal reproduction (1.5.5, no network required)

const { toChatCtx } = await import('@livekit/agents/dist/llm/provider_format/google.js');
const { ChatContext } = await import('@livekit/agents/dist/llm/chat_context.js');

const mk = (items) => { const c = ChatContext.empty(); for (const [role, content] of items) c.addMessage({ role, content }); return c; };

await toChatCtx(mk([['user','hi'], ['assistant','partial reply']]));
// → [user, model, user]  ✅ guard fires

await toChatCtx(mk([['user','hi'], ['assistant','partial reply'], ['user','']]));
// → [user, model]        ❌ ends with model → 400 INVALID_ARGUMENT

await toChatCtx(mk([['assistant','partial reply'], ['user','']]));
// → [model]              ❌ same

Suggested fix

-  if (injectDummyUserMessage && currentRole !== "user") {
+  if (injectDummyUserMessage && turns[turns.length - 1]?.role !== "user") {

Verified against the same three cases plus the normal path — all end with a user turn, and the non-interrupted case is unchanged:

✓ [user, model, user GOL] → [user → model → user]
✓ [model, user GOL]       → [model → user]
✓ [user, model]           → [user → model → user]   (unchanged)

Note for the Python implementation

livekit-agents/livekit/agents/llm/_provider_format/google.py has the same shape (if inject_dummy_user_message and current_role not in ("user", "tool")) and looks vulnerable to the identical case, though I have not reproduced it there.

Separately, and independent of this fix: the deterministic 400 INVALID_ARGUMENT is still marked retryable: true, so an unchanged payload is retried until the attempts are exhausted. Treating INVALID_ARGUMENT as non-retryable would turn a ~25s outage into an immediate, visible failure.

[arsenieciprian]

Confirmed in production — userland workaround, and one dead end worth flagging

We deployed a workaround based on the analysis above and it holds, which also confirms the root cause empirically.

Dead end first: chatCtx.copy({ excludeEmptyMessage: true }) does not help. The interrupted turn is stored as content: [""] — an array holding an empty string — so it isn't treated as an empty message and survives the copy. Anyone reaching for that option first (as we did) will see no change.

What works — filtering the empty user messages out in llmNode, which lets the framework's own injectDummyUserMessage guard fire as intended:

override async llmNode(
  chatCtx: llm.ChatContext,
  toolCtx: llm.ToolContext,
  modelSettings: voice.ModelSettings,
) {
  const isEmptyUser = (it: llm.ChatItem): boolean =>
    it.type === 'message' && it.role === 'user' && !(it.textContent ?? '').trim();
  if (!chatCtx.items.some(isEmptyUser)) return super.llmNode(chatCtx, toolCtx, modelSettings);
  const cleaned = llm.ChatContext.empty();
  cleaned.insert(chatCtx.items.filter((it) => !isEmptyUser(it)));
  return super.llmNode(cleaned, toolCtx, modelSettings);
}

Validation on live telephony calls (gemini-3.5-flash-lite, Vertex global, @livekit/agents 1.5.5):

	before workaround (24h)	after workaround (3 test calls)
barge-ins (playout completed with interrupt)	—	6
empty user messages produced	—	3
400 … model turn errors	84	0
lost turns (retries exhausted)	21	0
calls completing a booking	1 of 7 affected	3 of 3

Each test call interrupted the agent mid-sentence while it was proposing an address — the exact pattern that previously produced the retry loop and ~25s of dead air.

This is of course a userland patch around framework behaviour; the one-line change in provider_format/google.ts proposed in the previous comment is the proper fix, and we'd drop this immediately once it lands.