Nova Sonic Realtime Model: LLM goes silent after saying it will run a tool (tool triggers only after user input) #3664
Description
Issue Description
I’m using the AWS Nova Sonic Realtime model, and most of the time it works fine.
But sometimes (around 20–30% of tests), the agent says it’s going to run a tool, but the tool doesn’t actually trigger until the user speaks again.
In my case, it happens with the verify_eligibility_tool.
The agent asks for the user’s first name, last name, and date of birth, and after collecting them it says:
“Let me check your eligibility verification…”
Then it just stops. No tool call is made until the user says something again (like “hello?”), and only then the tool runs.
Example
Expected behavior
_USER: I’d like to check my eligibility.
[ASSISTANT]: Sure, please tell me your first name, last name, and date of birth.
USER: John Doe, 10/05/1985.
[TOOLUSE] {verify_eligibility_tool} runs.
[ASSISTANT]: Thank you. I’ve verified your eligibility.
When the issue happens
USER: I’d like to check my eligibility.
[ASSISTANT]: Sure, please tell me your first name, last name, and date of birth.
USER: John Doe, 10/05/1985.
[ASSISTANT]: Let me check your eligibility verification… (goes silent)
[TOOLUSE] {verify_eligibility_tool} runs._
Question
Could this delay or missed tool call be related to using tool_choice="auto"?
Is there a way to make the tool trigger right away after the model decides to use it?