Add comprehensive agentic memory API documentation for OpenSearch 3.3

_ml-commons-plugin/api/agentic-memory-apis/add-memory.md

@@ -10,10 +10,14 @@
 **Introduced 3.2**
 {: .label .label-purple }
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, join the discussion on the [OpenSearch forum](https://forum.opensearch.org/).    
-{: .warning}
 
-Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create a memory in one of the following modes (controlled by the `infer` parameter):
+Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create memories in two types:
+
+- **Conversation memory** -- Stores conversational messages between users and assistants. Can be processed (when `infer` is `true`) to extract facts or stored as raw messages.
+
+- **Data memory** -- Stores structured, non-conversational data such as agent state, checkpoints, or reference information.
+
+Memory processing modes (controlled by the `infer` parameter):
 
 - Fact memory -- A processed representation of the message. The large language model (LLM) associated with the memory container extracts and stores key factual information or knowledge from the original text.
 
@@ -33,41 +37,133 @@
 
 Field | Data type | Required/Optional | Description
 :--- | :--- | :--- | :---
-`messages` | List | Required | A list of messages. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`.
-`session_id` | String | Optional | The session ID associated with the memory.
-`agent_id` | String | Optional | The agent ID associated with the memory.
-`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true`. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message. 
+`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`.
+`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`.
+`memory_type` | String | Required | The type of memory: `conversation` or `data`.
+`namespace` | Object | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`).
+`metadata` | Object | Optional | Additional metadata for the memory (e.g., `status`, `branch`, custom fields).
+`session_id` | String | Optional | The session ID associated with the memory. Deprecated in favor of using `namespace.session_id`.
+`agent_id` | String | Optional | The agent ID associated with the memory. Deprecated in favor of using `namespace.agent_id`.
+`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true` for conversation memory, `false` for data memory. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message. 
 `tags` | Object | Optional | Custom metadata for the agentic memory.
 
-## Example request
+## Example requests
+
+### Conversation memory
+
+```json
+POST /_plugins/_ml/memory_containers/SdjmmpgBOh0h20Y9kWuN/memories
+{
+  "messages": [
+    {
+      "role": "user",
+      "content": "I'm Bob, I really like swimming."
+    },
+    {
+      "role": "assistant",
+      "content": "Cool, nice. Hope you enjoy your life."
+    }
+  ],
+  "namespace": {
+    "user_id": "bob"
+  },
+  "metadata": {
+    "status": "checkpoint",
+    "branch": {
+      "branch_name": "high",
+      "root_event_id": "228nadfs879mtgk"
+    }
+  },
+  "tags": {
+    "topic": "personal info"
+  },
+  "infer": true,
+  "memory_type": "conversation"
+}
+```
+
+### Data memory
 
 ```json
 POST /_plugins/_ml/memory_containers/SdjmmpgBOh0h20Y9kWuN/memories
 {
-    "messages": [
-        {"role": "user", "content": "Machine learning is a subset of artificial intelligence"}
-    ],
-    "session_id": "sess_789",
-    "agent_id": "agent_123",
-    "tags": {
-        "topic": "personal info"
+  "structured_data": {
+    "time_range": {
+      "start": "2025-09-11",
+      "end": "2025-09-15"
     }
+  },
+  "namespace": {
+    "agent_id": "testAgent1"
+  },
+  "metadata": {
+    "status": "checkpoint",
+    "anyobject": "abc"
+  },
+  "tags": {
+    "topic": "agent_state"
+  },
+  "infer": false,
+  "memory_type": "data"
+}
+```
+
+### Trace data memory
+
+```json
+POST /_plugins/_ml/memory_containers/SdjmmpgBOh0h20Y9kWuN/memories
+{
+  "structured_data": {
+    "tool_invocations": [
+      {
+        "tool_name": "ListIndexTool",
+        "tool_input": {
+          "filter": "*,-.plugins*"
+        },
+        "tool_output": "green  open security-auditlog-2025.09.17..."
+      }
+    ]
+  },
+  "namespace": {
+    "user_id": "bob",
+    "agent_id": "testAgent1",
+    "session_id": "123"
+  },
+  "metadata": {
+    "status": "checkpoint",
+    "branch": {
+      "branch_name": "high",
+      "root_event_id": "228nadfs879mtgk"
+    },
+    "anyobject": "abc"
+  },
+  "tags": {
+    "topic": "personal info",
+    "parent_memory_id": "o4-WWJkBFT7urc7Ed9hM",
+    "data_type": "trace"
+  },
+  "infer": false,
+  "memory_type": "conversation"
 }
 ```
 {% include copy-curl.html %}
 
-## Example response
+## Example responses
+
+### Conversation memory response
+
+```json
+{
+  "session_id": "XSEuiJkBeh2gPPwzjYVh",
+  "working_memory_id": "XyEuiJkBeh2gPPwzjYWM"
+}
+```
+
+### Data memory response
 
 ```json
 {
-    "results": [
-        {
-            "id": "T9jtmpgBOh0h20Y91WtZ",
-            "text": "Machine learning is a subset of artificial intelligence",
-            "event": "ADD"
-        }
-    ],
-    "session_id": "sess_789"
+  "working_memory_id": "Z8xeTpkBvwXRq366l0iA"
 }
 ```
 
@@ -77,8 +173,5 @@
 
 | Field           | Data type | Description                                                                                       |
 | :-------------- | :-------- | :------------------------------------------------------------------------------------------------ |
-| `results`       | List      | A list of memory entries returned by the request.                                                 |
-| `results.id`    | String    | The unique identifier for the memory entry.                                                       |
-| `results.text`  | String    | If `infer` is `false`, contains the stored text from the message. If `infer` is `true`, contains the extracted fact from the message.             |
-| `results.event` | String    | The type of event for the memory entry. For the Add Agentic Memory API, the event type is always `ADD`, indicating that the memory was added. |
-| `session_id`    | String    | The session ID associated with the memory.                                    |
+| `session_id`    | String    | The session ID associated with the memory (returned for conversation memory when a session is created or used). |
+| `working_memory_id` | String | The unique identifier for the created working memory entry. |

_ml-commons-plugin/api/agentic-memory-apis/create-memory-container.md

@@ -10,14 +10,13 @@ nav_order: 10
 **Introduced 3.2**
 {: .label .label-purple }
 
-This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, join the discussion on the [OpenSearch forum](https://forum.opensearch.org/).    
-{: .warning}
-
 Use this API to create a memory container to hold agentic memories. The container can have two model types associated with it:
 
 - A text embedding model for vectorizing the message so it can be searched. Use a text embedding model for dense vector embeddings or a sparse encoding model for sparse vector formats. If no embedding model is specified, messages are stored but cannot be used for vector-based searches.
 - A large language model (LLM) for reasoning over the message to produce factual or processed content. If no LLM is specified, messages are stored directly, without applying inference.
 
+**Note**: LLM connectors must support `system_prompt` and `user_prompt` parameters for agentic memory processing. The default `llm_result_path` is configured for Bedrock Converse API format (`"$.output.message.content[0].text"`).
+
 Once a memory container is created, you'll provide its `memory_container_id` to other APIs.
 
 ## Prerequisites
@@ -143,33 +142,110 @@ Field | Data type | Required/Optional | Description
 :--- | :--- | :--- | :---
 `name` | String | Required | The name of the memory container.
 `description` | String | Optional | The description of the memory container.
-`memory_storage_config` | Object | Optional | The memory storage configuration. See [the `memory_storage_config` object](#the-memory_storage_config-object).
+`configuration` | Object | Required | The memory container configuration. See [the `configuration` object](#the-configuration-object).
 
-### The memory_storage_config object
+### The configuration object
 
-The `memory_storage_config` object supports the following fields.
+The `configuration` object supports the following fields.
 
 Field | Data type | Required/Optional | Description
 :--- | :--- | :--- | :---
-`dimension` | Integer | Optional | The dimension of the embedding model. Required if `embedding_model_type` is `TEXT_EMBEDDING`.
-`embedding_model_id` | String | Optional | The embedding model ID.
 `embedding_model_type` | String | Optional | The embedding model type. Supported types are `TEXT_EMBEDDING` and `SPARSE_ENCODING`.
-`llm_model_id` | String | Optional | The LLM ID.
-`max_infer_size` | Integer | Optional | The maximum number of messages the LLM processes for inference in a single request. Valid values are 1--9, inclusive. Default is 5.
-`memory_index_name` | String | Optional | The name of the index in which to save messages, embeddings, and inferred facts. If not specified, a default index is automatically generated.
+`embedding_model_id` | String | Optional | The embedding model ID.
+`embedding_dimension` | Integer | Optional | The dimension of the embedding model. Required if `embedding_model_type` is `TEXT_EMBEDDING`.
+`llm_id` | String | Optional | The LLM model ID for processing and inference.
+`index_prefix` | String | Optional | Custom prefix for the memory indices. If not specified, a default prefix is used.
+`use_system_index` | Boolean | Optional | Whether to use system indices. Default is `true`.
+`strategies` | Array | Optional | Array of memory processing strategies. See [the `strategies` array](#the-strategies-array).
+`parameters` | Object | Optional | Global parameters for the memory container. See [the `parameters` object](#the-parameters-object).
+
+### The strategies array
+
+Each strategy in the `strategies` array supports the following fields.
+
+Field | Data type | Required/Optional | Description
+:--- | :--- | :--- | :---
+`type` | String | Required | The strategy type: `SEMANTIC`, `USER_PREFERENCE`, or `SUMMARY`.
+`namespace` | Array | Required | Array of namespace dimensions for organizing memories (e.g., `["user_id"]`, `["agent_id"]`).
+`configuration` | Object | Optional | Strategy-specific configuration. See [the strategy `configuration` object](#the-strategy-configuration-object).
+
+### The strategy configuration object
+
+Field | Data type | Required/Optional | Description
+:--- | :--- | :--- | :---
+`llm_result_path` | String | Optional | JSONPath to extract LLM results. Default is `"$.output.message.content[0].text"` for Bedrock Converse API format.
+
+### The parameters object
+
+Field | Data type | Required/Optional | Description
+:--- | :--- | :--- | :---
+`llm_result_path` | String | Optional | Global JSONPath for extracting LLM results from responses. Default is `"$.output.message.content[0].text"` for Bedrock Converse API format.
+
+## Example requests
+
+### Basic memory container
+
+```json
+POST /_plugins/_ml/memory_containers/_create
+{
+  "name": "agentic memory test",
+  "description": "Store conversations with semantic search and summarization",
+  "configuration": {
+    "embedding_model_type": "TEXT_EMBEDDING",
+    "embedding_model_id": "{{embedding_model_id}}",
+    "embedding_dimension": 1024,
+    "llm_id": "{{llm_id}}",
+    "strategies": [
+      {
+        "type": "SEMANTIC",
+        "namespace": ["user_id"]
+      }
+    ]
+  }
+}
+```
 
-## Example request
+### Advanced memory container with multiple strategies
 
 ```json
 POST /_plugins/_ml/memory_containers/_create
 {
-    "name": "Sparse memory container",
-    "description": "Store sparse conversations with semantic search",
-    "memory_storage_config": {
-        "llm_model_id": "bbphdJgB9L0Qb_M6ipnn",
-        "embedding_model_type": "SPARSE_ENCODING",
-        "embedding_model_id": "RodoX5gBfObQ5OgTHf1X"
+  "name": "agentic memory test",
+  "description": "Store conversations with semantic search and summarization",
+  "configuration": {
+    "embedding_model_type": "TEXT_EMBEDDING",
+    "embedding_model_id": "{{embedding_model_id}}",
+    "embedding_dimension": 1024,
+    "llm_id": "{{llm_id}}",
+    "index_prefix": "my_custom_prefix",
+    "use_system_index": false,
+    "strategies": [
+      {
+        "type": "SEMANTIC",
+        "namespace": ["agent_id"],
+        "configuration": {
+          "llm_result_path": "$.output.message.content[0].text"
+        }
+      },
+      {
+        "type": "USER_PREFERENCE",
+        "namespace": ["agent_id"],
+        "configuration": {
+          "llm_result_path": "$.output.message.content[0].text"
+        }
+      },
+      {
+        "type": "SUMMARY",
+        "namespace": ["agent_id"],
+        "configuration": {
+          "llm_result_path": "$.output.message.content[0].text"
+        }
+      }
+    ],
+    "parameters": {
+      "llm_result_path": "$.output.message.content[0].text"
     }
+  }
 }
 ```
 {% include copy-curl.html %}