Updating QueryPlanningTool documentation #11133

joshpalis · 2025-09-29T18:39:50Z

Description

Update documentation for QueryPlanningTool

Updates agent tools list with QueryPlanningTool
Updates QueryPlanningTool registration parameters documentation
Adds examples for QueryPlanningTool search_templates
Adds examples for Claud-4 and GPT-5 model registration

Issues Resolved

Part of #11026

Version

3.3

Frontend features

If you're submitting documentation for an OpenSearch Dashboards feature, add a video that shows how a user will interact with the UI step by step. A voiceover is optional.

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-09-29T18:39:58Z

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

owaiskazi19 · 2025-09-29T19:20:18Z

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

+}
+```
+
+Each `search_templates` array entry must have a `template_id` and `template_description`. The LLM will use the description as additional context to help it choose the best template to use when generating an OpenSearch DSL query based on the uses provided `query_text`. If the LLM determines that the provided search templates are not applicable to the given `query_text`, then the LLM will use a default match all query search template. It is important to note that the LLM will not directly render the final Opensearch DSL query with the chosen template, but rather use the template source as additional context to help it form the query.


Suggested change

Each `search_templates` array entry must have a `template_id` and `template_description`. The LLM will use the description as additional context to help it choose the best template to use when generating an OpenSearch DSL query based on the uses provided `query_text`. If the LLM determines that the provided search templates are not applicable to the given `query_text`, then the LLM will use a default match all query search template. It is important to note that the LLM will not directly render the final Opensearch DSL query with the chosen template, but rather use the template source as additional context to help it form the query.

Each `search_templates` list entry must have a `template_id` and `template_description`. The LLM will use the description as additional context to help it choose the best template to use when generating an OpenSearch DSL query based on the uses provided `query_text`. If the LLM determines that the provided search templates are not applicable to the given `query_text`, then the LLM will use a default match all query search template. It is important to note that the LLM will not directly render the final Opensearch DSL query with the chosen template, but rather use the template source as additional context to help it form the query.

owaiskazi19 · 2025-09-29T19:21:01Z

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

+}
+```
+
+Each `search_templates` array entry must have a `template_id` and `template_description`. The LLM will use the description as additional context to help it choose the best template to use when generating an OpenSearch DSL query based on the uses provided `query_text`. If the LLM determines that the provided search templates are not applicable to the given `query_text`, then the LLM will use a default match all query search template. It is important to note that the LLM will not directly render the final Opensearch DSL query with the chosen template, but rather use the template source as additional context to help it form the query.


Just default query, right? We don't have a default template for match all query

Suggested change

Each `search_templates` array entry must have a `template_id` and `template_description`. The LLM will use the description as additional context to help it choose the best template to use when generating an OpenSearch DSL query based on the uses provided `query_text`. If the LLM determines that the provided search templates are not applicable to the given `query_text`, then the LLM will use a default match all query search template. It is important to note that the LLM will not directly render the final Opensearch DSL query with the chosen template, but rather use the template source as additional context to help it form the query.

Each `search_templates` array entry must have a `template_id` and `template_description`. The LLM will use the description as additional context to help it choose the best template to use when generating an OpenSearch DSL query based on the uses provided `query_text`. If the LLM determines that the provided search templates are not applicable to the given `query_text`, then the LLM will use a default match all query. It is important to note that the LLM will not directly render the final Opensearch DSL query with the chosen template, but rather use the template source as additional context to help it form the query.

owaiskazi19 · 2025-09-29T19:22:28Z

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

 You can use any [OpenSearch agent type](https://opensearch.org/docs/latest/ml-commons-plugin/agents-tools/agents/) to run the `QueryPlanningTool`. The following example uses a `flow` agent, which runs a sequence of tools in order and returns the last tool's output.

-When registering the agent, you can override parameters set during model registration, such as `system_prompt`.
+When registering the agent, you can override parameters set during model registration, such as `query_planner_system_prompt` and `query_planner_user_prompt`. The following example registers an agent with a default `llmGenerated` generation_type.


Can we add a heading with 1. LLM Generated and then 2. User Defined Templates?

@joshpalis Normally, parameters are in snake_case. Since this was an experimental feature, is there time to make this llm_generated instead of llmGenerated so it follows this convention?

owaiskazi19 · 2025-09-29T19:23:41Z

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

+        "query_planner_user_prompt": "You are an OpenSearch Query DSL generation expert. Use the following index mapping to inform your query: ${parameters.index_mapping}. Based on this mapping, generate a query for the user's question: ${parameters.question}. Only use fields mentioned in the mapping.",
        "question": "Find all movies directed by John Doe",
        "index_mapping": "{\"properties\":{\"title\":{\"type\":\"text\"},\"director\":{\"type\":\"keyword\"},\"year\":{\"type\":\"integer\"}}}"
    }


You can also remove the experimental tag from next steps and update the neural search issue with opensearch-project/neural-search#1525

rithin-pullela-aws

Pointed out some minor concerns, we can correct these later as well.

rithin-pullela-aws · 2025-09-29T19:18:17Z

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

+Creating search templates :
+```json
+POST /_scripts/flower_species_search_template
+{
+    "script": {
+        "lang": "mustache",
+        "source": {
+            "from": "{{from}}{{^from}}0{{/from}}",
+            "size": "{{size}}{{^size}}10{{/size}}",
+            "query": {
+                "match": {
+                    "species": "{{species}}"
+                }
+            }
+        }
+    }
+}
+```
+
+```json
+POST /_scripts/flower_species_search_template_2
+{
+    "script": {
+        "lang": "mustache",
+        "source": {
+            "from": "{{from}}{{^from}}0{{/from}}",
+            "size": "{{size}}{{^size}}10{{/size}}",
+            "query": {
+                "term": {
+                    "species": "{{species}}"
+                }
+            }
+        }
+    }
+}
+```


Can we have more complicated examples? We might want to showcase 2 things

the LLM is capable to choosing the template based on the description

The Templates can be used when we have a really complicated use case that the LLM cannot generate the query itself or when we want determinism in the LLM response.

I can perhaps showcases a complicated use case in Agentic Search documentation, but we should at least show the LLM choosing between 2 different templates effectively. Here looks like both the templates are same with same description as well. Perhaps we can go with a little more complicated use case?

Sure, I can add a more complicated scenario, perhaps its better to utilize the same example you intend to use for Agentic Search documentation

rithin-pullela-aws · 2025-09-29T19:19:22Z

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

+            "description": "A general tool to answer any question",
+            "parameters": {
+                "model_id": "ANiSxJgBOh0h20Y9XYXl",
+                "response_filter": "$.output.message.content[0].text",


While adding response filter here totally works, we might want to add this in the model registration body under parameters (same effect, just cleaner approach when considering the Conversational Agent)

rithin-pullela-aws · 2025-09-29T19:21:54Z

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

 {
    "parameters": {
-        "user_prompt": "You are an OpenSearch Query DSL generation assistant, generate an OpenSearch Query DSL to retrieve the most relevant documents for the user provided natural language question: ${parameters.query_text}, please return the query dsl only, no other texts. Please don't use size:0, because that would limit the query to return no result. please return a query to find the most relevant documents related to users question. For example: {\"query\":{\"match\":{\"species\":\"setosa\"}}} \n",
+        "query_planner_user_prompt": "You are an OpenSearch Query DSL generation assistant, generate an OpenSearch Query DSL to retrieve the most relevant documents for the user provided natural language question: ${parameters.query_text}, please return the query dsl only, no other texts. Please don't use size:0, because that would limit the query to return no result. please return a query to find the most relevant documents related to users question. For example: {\"query\":{\"match\":{\"species\":\"setosa\"}}} \n",


Can we mention that we have a default system and user prompt? And perhaps remove this from the execute operation to make it less confusing? From 3.3 the user_prompt is meant to provide the information about question etc not the guidance about DSL generation like in this example. So maybe remove it altogether?

What execute example should we use here instead?

rithin-pullela-aws · 2025-09-29T19:22:59Z

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

    "parameters": {
-        "user_prompt": "You are an OpenSearch Query DSL generation assistant, generate an OpenSearch Query DSL to retrieve the most relevant documents for the user provided natural language question: ${parameters.query_text}, please return the query dsl only, no other texts. Please don't use size:0, because that would limit the query to return no result. please return a query to find the most relevant documents related to users question. For example: {\"query\":{\"match\":{\"species\":\"setosa\"}}} \n",
+        "query_planner_user_prompt": "You are an OpenSearch Query DSL generation assistant, generate an OpenSearch Query DSL to retrieve the most relevant documents for the user provided natural language question: ${parameters.query_text}, please return the query dsl only, no other texts. Please don't use size:0, because that would limit the query to return no result. please return a query to find the most relevant documents related to users question. For example: {\"query\":{\"match\":{\"species\":\"setosa\"}}} \n",
        "query_text": "How many iris flowers of type setosa are there?"


The query_text will be called question from 3.3

Signed-off-by: Joshua Palis <[email protected]>

…mments Signed-off-by: Joshua Palis <[email protected]>

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws · 2025-10-01T18:05:14Z

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

-{: .label .label-purple }
-
-Introduced in 3.2
+Introduced 3.3


We normally list the version when the feature is GA in the "introduced" label.

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

Signed-off-by: kolchfa-aws <[email protected]>

natebower

Editorial review

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md

Signed-off-by: Nathan Bower <[email protected]>

natebower

LGTM

joshpalis requested review from AMoo-Miki, dlvenable, epugh, kolchfa-aws, natebower and sumobrian as code owners September 29, 2025 18:39

github-actions bot assigned kolchfa-aws Sep 29, 2025

owaiskazi19 reviewed Sep 29, 2025

View reviewed changes

rithin-pullela-aws reviewed Sep 29, 2025

View reviewed changes

kolchfa-aws added Tech review PR: Tech review in progress release-notes PR: Include this PR in the automated release notes v3.3.0 labels Sep 29, 2025

joshpalis added 2 commits September 30, 2025 18:27

Updating QueryPlanningTool documentation

a5bc0a5

Signed-off-by: Joshua Palis <[email protected]>

Updating QueryPlanningTool registration parameters

8aa1698

Signed-off-by: Joshua Palis <[email protected]>

joshpalis force-pushed the doc branch from a4c3b9d to 8aa1698 Compare September 30, 2025 18:28

Updating documentation, removing experimental mentions, addressing co…

9c4da4b

…mments Signed-off-by: Joshua Palis <[email protected]>

owaiskazi19 mentioned this pull request Oct 1, 2025

[DOC] Documentation for Agentic Search GA #11026

Closed

4 tasks

kolchfa-aws added Doc review PR: Doc review in progress and removed Tech review PR: Tech review in progress labels Oct 1, 2025

Doc review

85d9a8c

Signed-off-by: Fanit Kolchina <[email protected]>

kolchfa-aws reviewed Oct 1, 2025

View reviewed changes

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md Outdated Show resolved Hide resolved

Update _ml-commons-plugin/agents-tools/tools/query-planning-tool.md

9762c1a

Signed-off-by: kolchfa-aws <[email protected]>

kolchfa-aws added Editorial review PR: Editorial review in progress and removed Doc review PR: Doc review in progress labels Oct 1, 2025

kolchfa-aws assigned natebower and unassigned kolchfa-aws Oct 1, 2025

natebower reviewed Oct 1, 2025

View reviewed changes

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md Outdated Show resolved Hide resolved

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md Outdated Show resolved Hide resolved

_ml-commons-plugin/agents-tools/tools/query-planning-tool.md Outdated Show resolved Hide resolved

Apply suggestions from code review

420e698

Signed-off-by: Nathan Bower <[email protected]>

natebower approved these changes Oct 1, 2025

View reviewed changes

natebower removed the Editorial review PR: Editorial review in progress label Oct 1, 2025

natebower merged commit b89023d into opensearch-project:main Oct 1, 2025
6 checks passed

Updating QueryPlanningTool documentation #11133

Updating QueryPlanningTool documentation #11133

Uh oh!

Conversation

joshpalis commented Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues Resolved

Version

Frontend features

Checklist

Uh oh!

github-actions bot commented Sep 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rithin-pullela-aws left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rithin-pullela-aws Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

natebower left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

natebower left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

joshpalis commented Sep 29, 2025 •

edited

Loading

rithin-pullela-aws Sep 29, 2025 •

edited

Loading