fix(responses): return error to model on hallucinated tool calls instead of silently exiting#6231
Merged
mattf merged 4 commits intoJul 8, 2026
Conversation
…ead of silently exiting When the LLM generates a tool call for a function name that does not match any registered server-side tool in a server-only agentic loop, OGX now feeds a structured error back to the model listing available tools, allowing it to self-correct. Previously the call was classified as a client-side function call causing the inference loop to exit silently with no error surfaced to the caller. A retry counter (_MAX_HALLUCINATED_TOOL_RETRIES=3) caps consecutive hallucinated attempts and exits with status "incomplete" to prevent infinite loops. Signed-off-by: Matteo Grimaldi <matteo@redhat.com> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Matteo Grimaldi <mgrimald@redhat.com>
matteo-grimaldi
requested review from
bbrowning,
cdoern,
franciscojavierarceo,
leseb,
mattf,
raghotham and
skamenan7
as code owners
Collaborator
|
@matteo-grimaldi great idea! please address ci failures. |
Existing tests unpacked 4 values from _separate_tool_calls but the hallucinated-tool-call change added a 5th (has_hallucinated_retries). Also fix import ordering to satisfy ruff's isort rule. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Matteo Grimaldi <mgrimald@redhat.com>
…ol-call-silent-exit
…st refactor Upstream PR ogx-ai#6227 changed create_openai_response to accept a single CreateResponseRequest object instead of individual keyword arguments. Update the two new tests to use the new calling convention. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Matteo Grimaldi <mgrimald@redhat.com>
Collaborator
|
@matteo-grimaldi couple more |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug: OGX Silently Exits Inference Loop on Unrecognized Tool Calls
Component: OGX Responses API (
ogx.providers.inline.responses.builtin.responses.streaming)Severity: High (agent run terminates silently — no error surfaced to the user or caller)
Type: Bug
Description
When the LLM generates a tool call for a function name that does not match any tool registered in the MCP servers, OGX classifies it as a "client-side function call" and exits the inference loop. Because there is no client waiting to handle the call, the agent run simply stops. No error is returned to the caller, no diagnostic is written to the response object, and the MCP sessions are torn down.
From the user's perspective the agent appears to hang or silently stop producing output mid-diagnosis.
Observed Behaviour
Pod:
ocp-troubleshooter-6db9d4f7c9-hpzfsTimestamp: 2026-06-16 12:25:44 UTC
Model:
nemotron/nemotron-3-nano-30b-a3bfinish_reason: "tool_calls"Service tier is None, setting to defaultModel called unrecognized tool; treating as a client-side function call. name=services_listExiting inference loop since there is a function (client-side) tool callGET stream disconnected, reconnecting in 1000ms.../v1/modelshealth checks continueThe model called
services_list({"namespace": "demo-app"}). This tool does not exist in any of the three registered MCP servers:pods_list_in_namespace,pods_get,pods_log,pods_delete, and other Kubernetes resource tools — but notservices_list.query_prometheus,query_prometheus_range.create_incident,list_incidents,get_incident,update_incident,add_work_note.The nemotron model hallucinated a plausible but non-existent tool name — a known failure mode with smaller models, especially when tool schemas contain many functions with similar naming patterns (e.g.
pods_list_in_namespace→ the model infersservices_listshould also exist).