feat: Add Anthropic advanced tool use features #3550

ryx2 · 2025-11-25T09:55:11Z

Summary

This PR adds support for Anthropic's new advanced tool use features from the advanced-tool-use-2025-11-20 beta, announced at https://www.anthropic.com/engineering/advanced-tool-use.

Features Added

Tool Search Tool (ToolSearchTool): Enables dynamic tool discovery without loading all definitions upfront. Supports both regex and BM25 search types. New defer_loading field on ToolDefinition marks tools for on-demand loading.
Programmatic Tool Calling (ProgrammaticCodeExecutionTool): Enhanced code execution that allows Claude to write Python code that calls your custom tools programmatically. New allowed_callers field on ToolDefinition specifies which tool types can call a tool.
Tool Use Examples: New input_examples field on ToolDefinition provides example inputs demonstrating correct tool usage patterns.

Changes

Added defer_loading, allowed_callers, and input_examples fields to ToolDefinition
Added same parameters to Tool class and @agent.tool/@agent.tool_plain decorators
Added ToolSearchTool and ProgrammaticCodeExecutionTool builtin tools
Updated Anthropic model to map new fields and add beta header automatically
Updated _map_server_tool_use_block to handle tool search responses
Added comprehensive tests

Test plan

Added unit tests for all new ToolDefinition fields
Added tests for Tool class with new parameters
Added tests for new builtin tools
Added tests for Anthropic model _map_tool_definition with new fields
Added tests for beta header auto-detection
Added tests for Agent integration with new parameters
All 31 new tests pass

🤖 Generated with Claude Code

This adds support for Anthropic's new advanced tool use features from the `advanced-tool-use-2025-11-20` beta: ## Tool Search Tool - New `ToolSearchTool` builtin tool with regex and BM25 search types - Enables dynamic tool discovery without loading all definitions upfront - New `defer_loading` field on `ToolDefinition` to mark tools for on-demand loading ## Programmatic Tool Calling - New `ProgrammaticCodeExecutionTool` builtin tool - New `allowed_callers` field on `ToolDefinition` to allow tools to be called from within code execution ## Tool Use Examples - New `input_examples` field on `ToolDefinition` to provide example inputs that demonstrate correct tool usage patterns All new fields are supported on: - `ToolDefinition` dataclass - `Tool` class - `@agent.tool` and `@agent.tool_plain` decorators - `FunctionToolset.add_function` method The Anthropic model automatically adds the `advanced-tool-use-2025-11-20` beta header when any of these features are used. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

…nced-tool-use

- Use cast() instead of type: ignore for new tool dict types in Anthropic model - Fix test type errors by casting results to dict[str, Any] - Add proper pyright ignore comments for private method access - Use underscore prefix for unused variables 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

The new defer_loading, allowed_callers, and input_examples fields are now included in ToolDefinition serialization, so update all affected test snapshots. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

DouweM · 2025-11-26T23:33:22Z

pydantic_ai_slim/pydantic_ai/models/anthropic.py

+                    )
+                if 'advanced-tool-use-2025-11-20' not in beta_features:
+                    beta_features.append('advanced-tool-use-2025-11-20')
+            elif isinstance(tool, ProgrammaticCodeExecutionTool):


We should not introduce a new tool type just for this window of time where Anthropic has multiple versions of code-execution. Instead, we should make our logic smarter to automatically use the newer code_execution version when advanced-tool-use is enabled / when allowed_callers is used.

DouweM · 2025-11-26T23:34:09Z

pydantic_ai_slim/pydantic_ai/builtin_tools.py

+class ToolSearchTool(AbstractBuiltinTool):
+    """A builtin tool that enables dynamic tool discovery without loading all definitions upfront.
+
+    Instead of consuming tokens on hundreds of tool definitions, Claude searches for relevant tools


This description should be generic, not Claude specific. If the details are Claude specific, the link to the docs below suffices.

DouweM · 2025-11-26T23:34:46Z

pydantic_ai_slim/pydantic_ai/builtin_tools.py

+
+    * Anthropic (with `advanced-tool-use-2025-11-20` beta)
+
+    See https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/tool-search-tool for more info.


This link should be on the same line as Anthropic, as we may add more providers in the future

DouweM · 2025-11-26T23:37:05Z

pydantic_ai_slim/pydantic_ai/builtin_tools.py

+    See https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/tool-search-tool for more info.
+    """
+
+    search_type: Literal['regex', 'bm25'] = 'regex'


We have to build this builtin tool class with the expectation that other providers will support this in the future as well. But we don't know what values they'll support here, or whether they'll support regex at all. So the default value here should always be a "neutral" value. I.e., let's add None to use the provider's default, and then document what that is for Anthropic.

DouweM · 2025-11-26T23:37:35Z

pydantic_ai_slim/pydantic_ai/builtin_tools.py

+
+    Supported by:
+
+    * Anthropic (with `advanced-tool-use-2025-11-20` beta)


We don't need to mention the beta, as we'd enable it automatically if this is used

DouweM · 2025-11-26T23:41:24Z

pydantic_ai_slim/pydantic_ai/tools.py

+    nested structures, and API-specific conventions. Each example must validate against
+    the tool's `parameters_json_schema`.
+
+    Note: this is currently only supported by Anthropic models with the `advanced-tool-use-2025-11-20` beta.


See above; don't mention the beta header, just enable it automatically

DouweM · 2025-11-26T23:56:11Z

pydantic_ai_slim/pydantic_ai/tools.py

+    See https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/tool-search-tool for more info.
+    """
+
+    allowed_callers: list[str] | None = None


This is way too Anthropic-specific to be on the generic ToolDefinition, especially since it uses a magic value that will change and users shouldn't need to know or have in their code anyway.

The goal is for Pydantic AI's representations to be generic so that we can make the same agent definition work with different providers, no matter how differently they may implement a feature.

Ideally it'd also be something that we can make work for providers that don't natively support the feature, for example by implementing our own "programmatic tool calling" using a run_python tool or something (see exploration in #2037, #3093).

So ideally we'd have a generically named boolean field like programmatically_callable (bad name because it's not clear that "programmatically" refers to by the LLM rather than by the dev, but you get the idea), and then document that AnthropicModel automatically respects it, or that you can use some Pydantic AI-specific feature (like a ProgrammaticToolCallingToolset) to make it work with non-Anthropic models.

In Anthropic's case, just setting that field on one of the tool definitions should then result in the code execution tool and beta header to be added.

A few more things:

Per https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling, this feature is only supported by specific models. So we should add a AnthropicModelProfile field to indicate support, and only auto-enable the feature (i.e. send the allowed_callers field + code_execution builtin tool + beta header) if the current model is one of those

We likely have to store the caller from the tool call part on ToolCallPart.provider_details: https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling#the-caller-field-in-responses, and send it back to the API when we send the tool result

It looks like we should set the container ID on the request that has the tool result so it's reused: https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling#step-3-provide-tool-result

We also have to consider these limitations from https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling#feature-incompatibilities:

Structured outputs: Tools with strict: true are not supported with programmatic calling

Tool choice: You cannot force programmatic calling of a specific tool via tool_choice

Parallel tool use: disable_parallel_tool_use: true is not supported with programmatic calling

Per https://platform.claude.com/docs/en/agents-and-tools/tool-use/programmatic-tool-calling#the-allowed-callers-field, allowed_callers can be ["direct", "code_execution_20250825"], but:

We recommend choosing either ["direct"] or ["code_execution_20250825"] for each tool rather than enabling both, as this provides clearer guidance to Claude for how best to use the tool.

Still, programmatically_callable: true does not clearly imply "not directly callable", so maybe we should send ["direct", "code_execution_20250825"] when it's set, and use programmatically_callable: 'only' to indicate just code_execution_20250825?

DouweM · 2025-11-27T00:00:04Z

pydantic_ai_slim/pydantic_ai/tools.py

    For MCP tools, this contains the `meta`, `annotations`, and `output_schema` fields from the tool definition.
    """

+    defer_loading: bool = False


Similar to the comment below (in the code, above in the chronological feed), I'm a little uncomfortable adding a field for an Anthropic-only feature that will likely come to other providers as well, but we don't know yet in what form, nor if Anthropic keeps this form since it's still in beta. Generally I'd say it's too early to add this field on ToolDefinition where we'll have to maintain it for backward compatibility, if it may turn out other providers take a different approach that would be incompatible, and we should wait until at least 2 providers have something to know how to do it.

But on the other hand, defer_loading seems like a reasonable not-Anthropic-specific name, and I could see how we could implement support for this in Pydantic AI using a new flag that will insert our own tool search tool.

So I'm OK with this field, but I'd want it to automatically add the ToolSearchTool when used. Similar to how specifying output_type=BinaryImage on OpenAIResponsesModel automatically adds the ImageGenerationTool, and how using strict=True on a ToolDefinition automatically adds the structured-outputs-2025-11-13 Anthropic beta header.

DouweM · 2025-11-27T00:00:24Z

pydantic_ai_slim/pydantic_ai/tools.py

+    See https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/code-execution-tool for more info.
+    """
+
+    input_examples: list[dict[str, Any]] | None = None


Should this just be examples to be more generic? Or do we expect output_examples as well?

DouweM · 2025-11-27T00:02:01Z

tests/models/test_anthropic_advanced_tools.py

Please move this to tests/models/anthropic, and add a test that actually triggers the programmatic tool calling feature to be used. I want to see a VCR cassette with the actual response we get from the API, both for streaming and non-streaming, and then what messages and streaming events correspond to that. You can look at test_anthropic_code_execution_tool and test_anthropic_code_execution_tool_stream for examples

DouweM · 2025-11-27T17:56:43Z

@ryx2 I added some notes on Programmatic Tool Calling: #3550 (comment). That looks non-trivial to implement. Can you please split the PR up into 3, one for each feature (PTC, search + defer, input examples) so it'll be easier to review and we can merge parts as they are ready instead of having to wait for everything?

ryx2 and others added 4 commits November 25, 2025 01:54

Merge remote-tracking branch 'upstream/main' into feat/anthropic-adva…

7894a30

…nced-tool-use

DouweM self-assigned this Nov 26, 2025

DouweM requested changes Nov 27, 2025

View reviewed changes

DouweM added the awaiting author revision label Nov 27, 2025

DouweM mentioned this pull request Nov 28, 2025

Support deferred loading of tools and discovery via tool search tool #3590

Open

DouweM added this to the 2025-12 milestone Nov 28, 2025


		* Anthropic (with `advanced-tool-use-2025-11-20` beta)

		See https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/tool-search-tool for more info.


		Supported by:

		* Anthropic (with `advanced-tool-use-2025-11-20` beta)

feat: Add Anthropic advanced tool use features #3550

Are you sure you want to change the base?

feat: Add Anthropic advanced tool use features #3550

Uh oh!

Conversation

ryx2 commented Nov 25, 2025

Summary

Features Added

Changes

Test plan

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

DouweM commented Nov 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants