fix(google): omit scheduling parameter for Vertex AI function responses #3694
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fixes ValueError when using RealtimeModel with Vertex AI where function
calls execute successfully but sending responses back to the model fails
with: 'scheduling parameter is not supported in Vertex AI.'
Error traceback:
File "livekit/plugins/google/realtime/realtime_api.py", line 732
await session.send_tool_response(function_responses=msg.function_responses)
File "google/genai/live.py", line 407, in send_tool_response
tool_response_dict = live_converters._LiveClientToolResponse_to_vertex(...)
File "google/genai/_live_converters.py", line 2203
raise ValueError('scheduling parameter is not supported in Vertex AI.')
Root cause: get_tool_results_for_realtime() was setting the 'scheduling'
parameter to WHEN_IDLE by default for all FunctionResponse objects,
including when vertexai=True. Vertex AI's converter explicitly rejects
any scheduling parameter in function responses.
Impact: Functions would execute successfully but Gemini never received
the results, causing conversation flow to break, repeated function calls,
or the model to get stuck.
Fix: Conditionally construct FunctionResponse objects - only include
'scheduling' and 'id' parameters when vertexai=False, matching the
pattern already used for the 'id' parameter.
Related: Similar to how 'id' is excluded for Vertex AI per:
https://github.com/googleapis/python-genai/blob/85e00bc/google/genai/_live_converters.py#L1435
|
|
|
issue is fixed in #3781 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes
ValueErrorwhen usingRealtimeModelwith Vertex AI where function calls execute successfully but sending responses back to the model fails with:scheduling parameter is not supported in Vertex AI.Root cause:
get_tool_results_for_realtime()was setting theschedulingparameter toWHEN_IDLEby default for allFunctionResponseobjects, including whenvertexai=True. Vertex AI's converter explicitly rejects any scheduling parameter in function responses.Impact: Functions would execute successfully but Gemini never received the results, causing conversation flow to break, repeated function calls, or the model to get stuck.
Fix: Conditionally construct FunctionResponse objects - only include
schedulingandidparameters whenvertexai=False, matching the pattern already used for theidparameter.Related: Similar to how
idis excluded for Vertex AI per:https://github.com/googleapis/python-genai/blob/85e00bc/google/genai/_live_converters.py#L1435