add option "--expand-tools-even-if-tool-choice-none" #2

okdshin · 2025-04-25T07:27:42Z

Add option to include tool definitions even when tool_choice is 'none'

Summary

This PR adds a new command-line option --expand-tools-even-if-tool-choice-none which allows including tool definitions in prompts even when tool_choice='none'.

Motivation

In the current implementation, when tool_choice is set to 'none', all tool definitions are removed from the request, preventing the model from seeing the tool schemas. This change enables a workflow where:

The model can be aware of available tools (via their definitions in the prompt)
But is not expected to use them automatically (since tool_choice='none')

This is useful for:

Models that need to "plan" about available tools before using them in subsequent requests
Cases where tool descriptions provide useful context even if the tools aren't used
Situations where you want the model to reference tool capabilities in its response without actually calling them

Implementation

Added a new CLI flag --expand-tools-even-if-tool-choice-none (default: False)
Modified the request validation in protocol.py to no longer remove tools when tool_choice='none'
Added the new parameter to OpenAIServingChat and passed it through from the API server

github-actions · 2025-04-25T07:27:51Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

…ength from tokenizer config (vllm-project#19660) Signed-off-by: Ye (Charlotte) Qi <[email protected]>

Signed-off-by: Chengji Yao <[email protected]> Co-authored-by: mgoin <[email protected]>

Signed-off-by: Andy Xie <[email protected]>

Signed-off-by: Russell Bryant <[email protected]>

) Signed-off-by: SzymonOzog <[email protected]>

…llm-project#19626) Signed-off-by: Nick Hill <[email protected]>

…9557)

…rts. (vllm-project#19652) Signed-off-by: Shawn Tan <[email protected]>

) Signed-off-by: Isotr0py <[email protected]>

Signed-off-by: Andy Xie <[email protected]>

…oject#19662) Signed-off-by: chaunceyjiang <[email protected]>

Signed-off-by: Bill Nell <[email protected]>

…h MiniMaxText01ForCausalLM) (vllm-project#19677) Signed-off-by: QscQ <[email protected]>

…19446) Signed-off-by: Russell Bryant <[email protected]>

… 100 (vllm-project#19563)

…t#19641) Signed-off-by: conroy-cheers <[email protected]>

Signed-off-by: David Xia <[email protected]>

Signed-off-by: nguyenhoangthuan99 <[email protected]>

Signed-off-by: Di Liu <[email protected]>

Signed-off-by: drisspg <[email protected]>

…er (vllm-project#19686) Signed-off-by: 刘全 <[email protected]> Co-authored-by: 刘全 <[email protected]>

Signed-off-by: Qiang Li <[email protected]>

…oject#19583)

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

…9851) Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

Signed-off-by: Andy Xie <[email protected]>

Signed-off-by: Thomas Parnell <[email protected]>

Signed-off-by: nie3e <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

…llm-project#19164) Signed-off-by: Chendi.Xue <[email protected]>

…-project#18777) Signed-off-by: Vlad Mihailescu <[email protected]>

…ts (vllm-project#19901) Signed-off-by: jiang1.li <[email protected]>

…project#19749)

Signed-off-by: wangxiyuan <[email protected]>

…t#19910) Signed-off-by: Rabin Adhikari <[email protected]>

Signed-off-by: Isotr0py <[email protected]>

Signed-off-by: 汪志鹏 <[email protected]>

Signed-off-by: okada shintarou <[email protected]>

This reverts commit 377f4ac3d47d67ca05ed2df6054aab6faec02b4d. Signed-off-by: okada shintarou <[email protected]>

Signed-off-by: okada shintarou <[email protected]>

Co-authored-by: Aaron Pham <[email protected]> Signed-off-by: okada shintarou <[email protected]>

Signed-off-by: okada shintarou <[email protected]>

okdshin force-pushed the specifiable_tools_expansion branch 5 times, most recently from ef2bf5f to 25aec8d Compare June 14, 2025 13:29

yeqcharlotte and others added 23 commits June 16, 2025 06:30

[Misc] Fix skipped max-model-len validation when deriving max model l…

b692e9c

…ength from tokenizer config (vllm-project#19660) Signed-off-by: Ye (Charlotte) Qi <[email protected]>

[TPU] support attention head dim smaller than 128 (vllm-project#19620)

a77aea5

Signed-off-by: Chengji Yao <[email protected]> Co-authored-by: mgoin <[email protected]>

[MISC] typo fix (vllm-project#19672)

26bc46e

Signed-off-by: Andy Xie <[email protected]>

[CI] Add mteb testing for rerank models (vllm-project#19344)

f40f763

[Docs] Move multiproc doc to v1 dir (vllm-project#19651)

8d12070

Signed-off-by: Russell Bryant <[email protected]>

[Kernel] GGUF MMVQ kernel for multiple input vectors (vllm-project#18754

dec66d2

) Signed-off-by: SzymonOzog <[email protected]>

[BugFix] Don't catch BaseException when dumping execute_model errors (v…

ee35e96

…llm-project#19626) Signed-off-by: Nick Hill <[email protected]>

[DOC] Add reasoning capability to vLLM streamlit code (vllm-project#1…

3e75069

…9557)

[Feature]:Allow for Granite MoE Hybrid models with _only_ shared expe…

4d54240

…rts. (vllm-project#19652) Signed-off-by: Shawn Tan <[email protected]>

[Bugfix] Fix TP inference for Flex attention backend (vllm-project#19657

1173804

) Signed-off-by: Isotr0py <[email protected]>

[MISC] bump huggingface_hub pkg to 0.33.0 (vllm-project#19547)

c3fec47

Signed-off-by: Andy Xie <[email protected]>

[Bugfix] fix missing 'finish_reason': null in streaming chat (vllm-pr…

836d4ce

…oject#19662) Signed-off-by: chaunceyjiang <[email protected]>

[Kernels] Use empty for modular MoE workspaces (vllm-project#19667)

5e5baa9

Signed-off-by: Bill Nell <[email protected]>

[Model] Add support for MiniMaxM1ForCausalLM (shares architecture wit…

387bdf0

…h MiniMaxText01ForCausalLM) (vllm-project#19677) Signed-off-by: QscQ <[email protected]>

[V1] Change return type on get_multimodal_embeddings() (vllm-project#…

90f9c2e

…19446) Signed-off-by: Russell Bryant <[email protected]>

[Quantization] Remove FP4 emulation; Fall-back to marlin for device <…

6bc7b57

… 100 (vllm-project#19563)

[Fix] Fall back to Gloo when NCCL backend is unavailable (vllm-projec…

0860087

…t#19641) Signed-off-by: conroy-cheers <[email protected]>

[doc] add project flag to gcloud TPU command (vllm-project#19664)

119f683

Signed-off-by: David Xia <[email protected]>

[Wheel Size] Only build FA2 8.0+PTX (vllm-project#19336)

0733495

[Frontend] add chunking audio for > 30s audio (vllm-project#19597)

ede5c4e

Signed-off-by: nguyenhoangthuan99 <[email protected]>

[DOC] fix doc typos (vllm-project#19600)

5b3ad5e

Signed-off-by: Di Liu <[email protected]>

Fixes IMA for TP w/ flex-attention (vllm-project#19712)

ddfed31

Signed-off-by: drisspg <[email protected]>

[Core] add remove_seq_from_computed_blocks_tracker to BlockSpaceManag…

5c76b9c

…er (vllm-project#19686) Signed-off-by: 刘全 <[email protected]> Co-authored-by: 刘全 <[email protected]>

qli88 and others added 15 commits June 20, 2025 12:43

[Bugfix] Enable PP with AITER+V1 (vllm-project#19822)

e3a3e4d

Signed-off-by: Qiang Li <[email protected]>

[Bugfix][Ray] Set the cuda context eagerly in the ray worker (vllm-pr…

5e666f7

…oject#19583)

[Misc] update cuda version (vllm-project#19526)

089a306

Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

[Misc] refactor example - openai_transcription_client (vllm-project#1…

e384f2f

…9851) Signed-off-by: reidliu41 <[email protected]> Co-authored-by: reidliu41 <[email protected]>

[Kernel] correct cpu worker function parameter type (vllm-project#19745)

71d1219

Signed-off-by: Andy Xie <[email protected]>

[Fix] import regex instead of re (vllm-project#19875)

7771d1d

Signed-off-by: Thomas Parnell <[email protected]>

[Model] GPT2ForSequenceClassification model (vllm-project#19663)

f1e840e

Signed-off-by: nie3e <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

[custom_op][vllm-plugin] update custom_op class to use op_registry (v…

7e8977f

…llm-project#19164) Signed-off-by: Chendi.Xue <[email protected]>

Export NaNs in logits to scheduler_stats if output is corrupted (vllm…

2e3e3c8

…-project#18777) Signed-off-by: Vlad Mihailescu <[email protected]>

[CPU][CI] Fallback sliding window to v0 and fix CPU pooling model tes…

79f2f1c

…ts (vllm-project#19901) Signed-off-by: jiang1.li <[email protected]>

[Kernel] mark TorchSDPABackend swap_blocks NotImplementedError (vllm-…

71baf85

…project#19749)

[Misc] Clean up useless code (vllm-project#19889)

e773a9e

Signed-off-by: wangxiyuan <[email protected]>

Fix: Check the type of params to be a Sequence not list. (vllm-projec…

8ca81bb

…t#19910) Signed-off-by: Rabin Adhikari <[email protected]>

[Bugfix] Fix bnb 8bit model weights loading (vllm-project#19917)

6f170f1

Signed-off-by: Isotr0py <[email protected]>

[New model support]Support Tarsier2 (vllm-project#19887)

c3bf9ba

Signed-off-by: 汪志鹏 <[email protected]>

okdshin force-pushed the specifiable_tools_expansion branch from 35649de to f2bc427 Compare June 21, 2025 14:36

okdshin added 8 commits June 21, 2025 23:59

add option "--expand-tools-even-if-tool-choice-none"

ba347cd

Signed-off-by: okada shintarou <[email protected]>

fix format

5b1b5e7

Signed-off-by: okada shintarou <[email protected]>

fix format

38eea42

Signed-off-by: okada shintarou <[email protected]>

remove the new option and change the default behaviour

bc031d9

Signed-off-by: okada shintarou <[email protected]>

Revert "remove the new option and change the default behaviour"

b5e4164

This reverts commit 377f4ac3d47d67ca05ed2df6054aab6faec02b4d. Signed-off-by: okada shintarou <[email protected]>

update help

e3733a9

Signed-off-by: okada shintarou <[email protected]>

add warn

1ffd7f0

Signed-off-by: okada shintarou <[email protected]>

fix docs

e282ebb

Signed-off-by: okada shintarou <[email protected]>

okdshin force-pushed the specifiable_tools_expansion branch from 9559897 to e282ebb Compare June 21, 2025 15:00

okdshin and others added 3 commits June 22, 2025 00:14

fix format

b51c7b9

Signed-off-by: okada shintarou <[email protected]>

fix format

0a0920a

Signed-off-by: okada shintarou <[email protected]>

Apply suggestions from code review

90eabb2

Co-authored-by: Aaron Pham <[email protected]> Signed-off-by: okada shintarou <[email protected]>

okdshin force-pushed the specifiable_tools_expansion branch from 8750191 to 90eabb2 Compare June 22, 2025 07:56

remove assert

8a45fa8

Signed-off-by: okada shintarou <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add option "--expand-tools-even-if-tool-choice-none" #2

add option "--expand-tools-even-if-tool-choice-none" #2

Uh oh!

okdshin commented Apr 25, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Apr 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

68 participants

add option "--expand-tools-even-if-tool-choice-none" #2

Are you sure you want to change the base?

add option "--expand-tools-even-if-tool-choice-none" #2

Uh oh!

Conversation

okdshin commented Apr 25, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add option to include tool definitions even when tool_choice is 'none'

Summary

Motivation

Implementation

Uh oh!

github-actions bot commented Apr 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

68 participants

okdshin commented Apr 25, 2025 •

edited by github-actions bot

Loading