Add standard blueprint for vector search #3659

mingshl · 2025-03-14T16:29:06Z

Description

Earlier, we have the these blueprint for embedding model which has pre and post processing function to format the model input and output format, so that it can be use for neural search query.
But now that we have ml inference processor that allow users to map the model input and output format, then user can register connector and model without pre and post processing function, We want to provide a new set of blueprint for embedding model with no pre and post processing function.

Related Issues

#3619

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Mingshi Liu <[email protected]>

...rence_blueprints/standard_blueprints/bedrock_connector_titan_embedding_standard_blueprint.md

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

brianf-aws

Left some minor comments, this is really great info to share!

brianf-aws · 2025-03-14T17:12:17Z

docs/remote_inference_blueprints/standard_blueprints/README.md

+# Model Blueprints for Vector Search
+
+OpenSearch provides two approaches for implementing embedding models, depending on your needs:
+


Suggested change

- Standard Blueprints

- Legacy Blueprints

This way they can follow what is the two approaches I get what you mean but I had to think a bit before I understood the idea.

you got it the other way around. the new blueprint without pre and post processing function is called Standard blueprint now. The earlier blueprint which has pre and post processing functions are called legacy blueprint

Oh I see so one is or ML inference processor and one is for connector (just using prediction)?

no, both are using connectors. One is without pre and post processing functions, but the other has pre or post processing functions

docs/remote_inference_blueprints/standard_blueprints/README.md

...s/standard_blueprints/bedrock_connector_cohere_cohere.embed-english-v3_standard_blueprint.md

...ndard_blueprints/bedrock_connector_cohere_cohere.embed-multilingual-v3_standard_blueprint.md

...rence_blueprints/standard_blueprints/bedrock_connector_titan_embedding_standard_blueprint.md

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

brianf-aws · 2025-03-14T17:37:15Z

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

+This is recommended for models to use the ML inference processor to handle input/output mapping.
+Note that if using a model that requires pre and post processing functions, you must provide the functions in the blueprint. Please refer to legacy blueprint: [Cohere Embedding Connector Blueprint for image embedding mode](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/cohere_connector_image_embedding_blueprint.md)
+
+- embed-english-v3.0 1024


lets be careful user might think the model name is embed-english-v3.0 1024 and not embed-english-v3.0

1024 is the token length I think

So i looked up the models and saw that these are dimension sizes. This is pretty important info to communicate, when user wants to do semantic search.

Please see here
https://arc.net/l/quote/ekkxjzmv

brianf-aws · 2025-03-14T17:40:27Z

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

+}
+```
+
+## 2. Create a connector 


Lets create a connector for managed too

cohere is not AWS service, user can use the same connector in open source and also on AWS, if you look at the existing legacy blueprint, you will find out it's the same

Got it, Lets link an example since they would need a secret Arn and a role Arn from my understanding (Using this as a reference https://github.com/opensearch-project/ml-commons/blob/main/docs/tutorials/aws/semantic_search_with_cohere_embedding_model.md#42-create-connector)

Yes, third-party platforms would require both secret arn and role arn on managed service: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ml-external-connector.html#connector-external-create

brianf-aws · 2025-03-14T17:42:47Z

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

+    "cohere_key": "<ENTER_COHERE_API_KEY_HERE>"
+  },
+  "parameters": {
+    "model": "<ENTER_MODEL_NAME_HERE>", // Choose a Model from the provided list above


This part is tricky to communicate. If user copys and paste they might not realize this comment cant be parsed. Maybe on the top we can say something along the lines of; for ENTER_MODEL_NAME_HERE make sure to choose embed-english-v3.0 or embed-english-v2.0

added embed-english-v3.0 as an example

Oh is this in a future commit? I dont see it on my end 😅

brianf-aws · 2025-03-14T17:46:24Z

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

+POST /_plugins/_ml/models/<MODEL_ID_HERE>/_predict
+{
+  "parameters": {
+    "images": ["data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAYCAYAAAD6S912AAAMP2lDQ1BJQ0MgUHJvZmlsZQAASImVVwdYU8kWnluSkEBoAQSkhN4EESkBpITQQu8INkISIJQQA0HFjiwquBZULGBDV0UUrIDYETuLYu8LKgrKuliwK29SQNd95XuTb+78+efMf86cmVsGALWTHJEoG1UHIEeYL44J8qOPT0qmk3qAMiACHfiz5HDzRMyoqDAAy1D79/LuJkCk7TV7qdY/+/9r0eDx87gAIFEQp/LyuDkQHwQAr+KKxPkAEKW82bR8kRTDCrTEMECIF0lxuhxXSXGqHO+V2cTFsCBuBUBJhcMRpwOgegXy9AJuOtRQ7YfYUcgTCAFQo0PsnZOTy4M4BWJraCOCWKrPSP1BJ/1vmqnDmhxO+jCWz0VWlPwFeaJszoz/Mx3/u+RkS4Z8WMKqkiEOjpHOGebtdlZuqBSrQNwnTI2IhFgT4g8CnsweYpSSIQmOl9ujBtw8FswZXGWAOvI4/qEQG0AcKMyOCFPwqWmCQDbEcIeg0wX57DiIdSFexM8LiFXYbBbnxih8oQ1pYhZTwZ/niGV+pb4eSrLimQr91xl8tkIfUy3MiEuEmAKxeYEgIQJiVYgd8rJiQxU24wozWBFDNmJJjDR+c4hj+MIgP7k+VpAmDoxR2Jfm5A3NF9ucIWBHKPD+/Iy4YHl+sFYuRxY/nAt2hS9kxg/p8PPGhw3Nhcf3D5DPHevhC+NjFTofRPl+MfKxOEWUHaWwx0352UFS3hRi57yCWMVYPCEfbki5Pp4myo+Kk8eJF2ZyQqLk8eDLQRhgAX9ABxJYU0EuyASC9r7GPvhP3hMIOEAM0gEf2CuYoRGJsh4hvMaCQvAnRHyQNzzOT9bLBwWQ/zrMyq/2IE3WWyAbkQWeQpwDQkE2/C+RjRIOe0sATyAj+Id3DqxcGG82rNL+f88Psd8ZJmTCFIxkyCNdbciSGED0JwYTA4k2uD7ujXviYfDqC6sTzsDdh+bx3Z7wlNBBeES4Qegk3JkiKBL/FGU46IT6gYpcpP6YC9wSarrgfrgXVIfKuA6uD+xxZ+iHiftAzy6QZSnilmaF/pP232bww2oo7MiOZJQ8guxLtv55pKqtqsuwijTXP+ZHHmvqcL5Zwz0/+2f9kH0ebEN/tsQWYQewc9gp7AJ2FGsEdOwE1oS1YcekeHh3PZHtriFvMbJ4sqCO4B/+hlZWmsk8x1rHXscv8r58/nTpMxqwckUzxIL0jHw6E74R+HS2kOswiu7k6OQMgPT9In98vYmWvTcQnbbv3II/APA6MTg4eOQ7F3ICgH1u8PY//J2zZsBXhzIA5w9zJeICOYdLLwT4lFCDd5oeMAJmwBrOxwm4Ak/gCwJACIgEcSAJTIbRZ8B9LgbTwCwwH5SAMrAcrAbrwSawFewEe8B+0AiOglPgLLgEroAb4B7cPd3gBegH78BnBEFICBWhIXqIMWKB2CFOCAPxRgKQMCQGSUJSkHREiEiQWcgCpAwpR9YjW5AaZB9yGDmFXEA6kDtIF9KLvEY+oRiqgmqhhqglOhploEw0FI1DJ6Hp6FS0EC1Gl6Jr0Wp0N9qAnkIvoTfQTvQFOoABTBnTwUwwe4yBsbBILBlLw8TYHKwUq8CqsTqsGa7zNawT68M+4kSchtNxe7iDg/F4nItPxefgS/D1+E68AW/Fr+FdeD/+jUAlGBDsCB4ENmE8IZ0wjVBCqCBsJxwinIH3UjfhHZFI1CFaEd3gvZhEzCTOJC4hbiDWE08SO4iPiQMkEkmPZEfyIkWSOKR8UglpHWk36QTpKqmb9EFJWclYyUkpUClZSahUpFShtEvpuNJVpWdKn8nqZAuyBzmSzCPPIC8jbyM3ky+Tu8mfKRoUK4oXJY6SSZlPWUupo5yh3Ke8UVZWNlV2V45WFijPU16rvFf5vHKX8kcVTRVbFZbKRBWJylKVHSonVe6ovKFSqZZUX2oyNZ+6lFpDPU19SP2gSlN1UGWr8lTnqlaqNqheVX2pRlazUGOqTVYrVKtQO6B2Wa1Pnaxuqc5S56jPUa9UP6x+S31Ag6YxRiNSI0djicYujQsaPZokTUvNAE2eZrHmVs3Tmo9pGM2MxqJxaQto22hnaN1aRC0rLbZWplaZ1h6tdq1+bU1tZ+0E7enaldrHtDt1MB1LHbZOts4ynf06N3U+jTAcwRzBH7F4RN2IqyPe647U9dXl65bq1uve0P2kR9cL0MvSW6HXqPdAH9e31Y/Wn6a/Uf+Mft9IrZGeI7kjS0fuH3nXADWwNYgxmGmw1aDNYMDQyDDIUGS4zvC0YZ+RjpGvUabRKqPjRr3GNGNvY4HxKuMTxs/p2nQmPZu+lt5K7zcxMAk2kZhsMWk3+WxqZRpvWmRab/rAjGLGMEszW2XWYtZvbmwebj7LvNb8rgXZgmGRYbHG4pzFe0sry0TLhZaNlj1WulZsq0KrWqv71lRrH+up1tXW122INgybLJsNNldsUVsX2wzbStvLdqidq53AboNdxyjCKPdRwlHVo27Zq9gz7Qvsa+27HHQcwhyKHBodXo42H508esXoc6O/Obo4Zjtuc7w3RnNMyJiiMc1jXjvZOnGdKp2uj6WODRw7d2zT2FfOds58543Ot11oLuEuC11aXL66urmKXetce93M3VLcqtxuMbQYUYwljPPuBHc/97nuR90/erh65Hvs9/jL094zy3OXZ884q3H8cdvGPfYy9eJ4bfHq9KZ7p3hv9u70MfHh+FT7PPI18+X5bvd9xrRhZjJ3M1/6OfqJ/Q75vWd5sGazTvpj/kH+pf7tAZoB8QHrAx4GmgamB9YG9ge5BM0MOhlMCA4NXhF8i23I5rJr2P0hbiGzQ1pDVUJjQ9eHPgqzDROHNYej4SHhK8PvR1hECCMaI0EkO3Jl5IMoq6ipUUeiidFR0ZXRT2PGxMyKORdLi50Suyv2XZxf3LK4e/HW8ZL4lgS1hIkJNQnvE/0TyxM7x48eP3v8pST9JEFSUzIpOSF5e/LAhIAJqyd0T3SZWDLx5iSrSdMnXZisPzl78rEpalM4Uw6kEFISU3alfOFEcqo5A6ns1KrUfi6Lu4b7gufLW8Xr5Xvxy/nP0rzSytN60r3SV6b3ZvhkVGT0CViC9YJXmcGZmzLfZ0Vm7cgazE7Mrs9RyknJOSzUFGYJW3ONcqfndojsRCWizqkeU1dP7ReHirfnIXmT8pryteCHfJvEWvKLpKvAu6Cy4MO0hGkHpmtMF05vm2E7Y/GMZ4WBhb/NxGdyZ7bMMpk1f1bXbObsLXOQOalzWuaazS2e2z0vaN7O+ZT5WfN/L3IsKi96uyBxQXOxYfG84se/BP1SW6JaIi65tdBz4aZF+CLBovbFYxevW/ytlFd6scyxrKLsyxLukou/jvl17a+DS9OWti9zXbZxOXG5cPnNFT4rdpZrlBeWP14ZvrJhFX1V6aq3q6esvlDhXLFpDWWNZE3n2rC1TevM1y1f92V9xvoblX6V9VUGVYur3m/gbbi60Xdj3SbDTWWbPm0WbL69JWhLQ7VldcVW4taCrU+3JWw79xvjt5rt+tvLtn/dIdzRuTNmZ2uNW03NLoNdy2rRWklt7+6Ju6/s8d/TVGdft6Vep75sL9gr2ft8X8q+m/tD97ccYByoO2hxsOoQ7VBpA9Iwo6G/MaOxsympqeNwyOGWZs/mQ0ccjuw4anK08pj2sWXHKceLjw+eKDwxcFJ0su9U+qnHLVNa7p0ef/p6a3Rr+5nQM+fPBp49fY557sR5r/NHL3hcOHyRcbHxkuulhjaXtkO/u/x+qN21veGy2+WmK+5XmjvGdRy/6nP11DX/a2evs69fuhFxo+Nm/M3btybe6rzNu91zJ/vOq7sFdz/fm3efcL/0gfqDiocGD6v/sPmjvtO181iXf1fbo9hH9x5zH794kvfkS3fxU+rTimfGz2p6nHqO9gb2Xnk+4Xn3C9GLz30lf2r8WfXS+uXBv3z/ausf39/9Svxq8PWSN3pvdrx1ftsyEDXw8F3Ou8/vSz/ofdj5kfHx3KfET88+T/tC+rL2q83X5m+h3+4P5gwOijhijuxTAIMVTUsD4PUOAKhJANDg+YwyQX7+kxVEfmaVIfCfsPyMKCuuANTB7/foPvh1cwuAvdvg8Qvqq00EIIoKQJw7QMeOHa5DZzXZuVJaiPAcsDn6a2pOKvg3RX7m/CHun1sgVXUGP7f/ApgGfHDsriOAAAAAimVYSWZNTQAqAAAACAAEARoABQAAAAEAAAA+ARsABQAAAAEAAABGASgAAwAAAAEAAgAAh2kABAAAAAEAAABOAAAAAAAAAJAAAAABAAAAkAAAAAEAA5KGAAcAAAASAAAAeKACAAQAAAABAAAAFKADAAQAAAABAAAAGAAAAABBU0NJSQAAAFNjcmVlbnNob3QbxQoxAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB1GlUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iWE1QIENvcmUgNi4wLjAiPgogICA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiPgogICAgICA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIgogICAgICAgICAgICB4bWxuczpleGlmPSJodHRwOi8vbnMuYWRvYmUuY29tL2V4aWYvMS4wLyI+CiAgICAgICAgIDxleGlmOlBpeGVsWURpbWVuc2lvbj4yNDwvZXhpZjpQaXhlbFlEaW1lbnNpb24+CiAgICAgICAgIDxleGlmOlBpeGVsWERpbWVuc2lvbj4yMDwvZXhpZjpQaXhlbFhEaW1lbnNpb24+CiAgICAgICAgIDxleGlmOlVzZXJDb21tZW50PlNjcmVlbnNob3Q8L2V4aWY6VXNlckNvbW1lbnQ+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+CjwveDp4bXBtZXRhPgpKZgjLAAAAHGlET1QAAAACAAAAAAAAAAwAAAAoAAAADAAAAAwAAAETEHSxzgAAAN9JREFUSA1ilJNX+s9ARcA4aiDFoTnUw5CRgUHFTIlBXFmMgV+cj+H///8Mbx+9Y3j94A3D0+vPGH59/40RRDi9zC3IxWAdbckgKieMoQkk8O7Ze4Z9cw4x/PzyE0Uep4HOafYMEiriYMUgzU+vPQO7UEJVgkFMQQQs/uHlR4bd0/ahuBSrgTLaUgz28TZgTY8uP2Y4svQEw/9/iAxlGmjEoGapApY/suQYw8NLT8BsEIHVQANPXQZtR02woo2dWxm+vP0K1wBisHKwMgTX+TMwszAx3Dx2h+HMhnNweQAAAAD//wm7Z2wAAAFtSURBVGOUk1f6z4AG7BKsGWS1pBl+ff/NsLp+PZoshOtb5snAJ8LL8PbJO4Ydk/bA1TBiM9A5zZ5BQkWc4c/vvwwv7ryAK0ZmiCmIMbBxsjJ8ePWJYWvPDrgUXgPhqvAwSDLw399/DJd2X8VjHAPD1/dfGR6cfwRXg9WFTil2DJJqEgzfP/9gWNe8Ca6YGAZWA00DjRnULJUZGIDRtapuHcPvn3+IMQusBquB6jaqDCZ+hmAFl3ZdYbi85xqKgdKakgzWkZYMDIwMDA8vPWY4ufo0XB6rgezcbAzexR4MnDwcDP///We4cfQ2w6OLjxmYWJgYRBVEGDTt1BnYudjAhhxbeZLh/tmH+A0EycrrywJdYcHAyAR0Bg7w5PozhkMLj4IthSnB6kKYpLCcEIOJvxGDsIwgAyMjwuCvH78x3DxyG4xBKQEZ4DUQppCVnYWBT5wP6Fomhu8fvjJ8/fgdHGEweWSaKAORNRBiAwBghdPpiOOz5gAAAABJRU5ErkJggg=="]


Oh noticing the format is a bit different here? before we expressed only the base64 string but this one has the data URI scheme. I wonder does the model only work with the URI?

I tried to use the same example from the legacy blueprint: https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/cohere_connector_image_embedding_blueprint.md would you suggest different format? I can try it out on my side

Yep lets test the regular base64 string without the URI

...s/standard_blueprints/bedrock_connector_cohere_cohere.embed-english-v3_standard_blueprint.md

docs/remote_inference_blueprints/standard_blueprints/README.md

ohltyler · 2025-03-14T18:15:28Z

...s/standard_blueprints/bedrock_connector_cohere_cohere.embed-english-v3_standard_blueprint.md

+
+## 2. Create connector for Amazon Bedrock:
+
+If you are using self-managed Opensearch, you should supply AWS credentials:


Thinking out loud, rather than doubling the number of blueprints based on AWS vs. self-managed, can we just persist one, and have a single call-out to the documentation describing how it is set up differently between AWS and self-managed? That should greatly reduce the amount of content.

ummm we already have AWS doc that talks about how to set up connectors on AWS. But I keep this just to be consist with legacy blueprints

Right exactly, was thinking to have just a single blueprint and one callout on the different ways to handle just the credential field, just as a general improvement over the legacy blueprints.

ohltyler · 2025-03-14T18:15:56Z

...s/standard_blueprints/bedrock_connector_cohere_cohere.embed-english-v3_standard_blueprint.md

+## 3. Create model group:
+
+```json
+POST /_plugins/_ml/model_groups/_register
+{
+    "name": "remote_model_group_cohere",
+    "description": "model group for cohere models"
+}
+```
+
+Sample response:
+```json
+{
+  "model_group_id": "IMobmY8B8aiZvtEZeO_i",
+  "status": "CREATED"
+}
+```


Can this be removed, now that there is default model groups and these steps aren't required?

model group is not required. but it's recommended for users to better manager their models, I will add the optional to 3. Create model group(Optional):

+1. Its good to add this step as optional to reduce the steps.

brianf-aws

I left some comments for review, and once addressed, I believe everything will be set and ready to go!

brianf-aws · 2025-03-17T01:48:04Z

docs/remote_inference_blueprints/standard_blueprints/README.md

+# Model Blueprints for Vector Search
+
+OpenSearch provides two approaches for implementing embedding models, depending on your needs:
+


Oh I see so one is or ML inference processor and one is for connector (just using prediction)?

...s/standard_blueprints/bedrock_connector_cohere_cohere.embed-english-v3_standard_blueprint.md

brianf-aws · 2025-03-17T01:53:48Z

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

+This is recommended for models to use the ML inference processor to handle input/output mapping.
+Note that if using a model that requires pre and post processing functions, you must provide the functions in the blueprint. Please refer to legacy blueprint: [Cohere Embedding Connector Blueprint for image embedding mode](https://github.com/opensearch-project/ml-commons/blob/main/docs/remote_inference_blueprints/cohere_connector_image_embedding_blueprint.md)
+
+- embed-english-v3.0 1024


So i looked up the models and saw that these are dimension sizes. This is pretty important info to communicate, when user wants to do semantic search.

Please see here
https://arc.net/l/quote/ekkxjzmv

brianf-aws · 2025-03-17T01:54:49Z

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

+    "cohere_key": "<ENTER_COHERE_API_KEY_HERE>"
+  },
+  "parameters": {
+    "model": "<ENTER_MODEL_NAME_HERE>", // Choose a Model from the provided list above


Oh is this in a future commit? I dont see it on my end 😅

brianf-aws · 2025-03-17T01:58:43Z

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

+POST /_plugins/_ml/models/<MODEL_ID_HERE>/_predict
+{
+  "parameters": {
+    "images": ["data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAYCAYAAAD6S912AAAMP2lDQ1BJQ0MgUHJvZmlsZQAASImVVwdYU8kWnluSkEBoAQSkhN4EESkBpITQQu8INkISIJQQA0HFjiwquBZULGBDV0UUrIDYETuLYu8LKgrKuliwK29SQNd95XuTb+78+efMf86cmVsGALWTHJEoG1UHIEeYL44J8qOPT0qmk3qAMiACHfiz5HDzRMyoqDAAy1D79/LuJkCk7TV7qdY/+/9r0eDx87gAIFEQp/LyuDkQHwQAr+KKxPkAEKW82bR8kRTDCrTEMECIF0lxuhxXSXGqHO+V2cTFsCBuBUBJhcMRpwOgegXy9AJuOtRQ7YfYUcgTCAFQo0PsnZOTy4M4BWJraCOCWKrPSP1BJ/1vmqnDmhxO+jCWz0VWlPwFeaJszoz/Mx3/u+RkS4Z8WMKqkiEOjpHOGebtdlZuqBSrQNwnTI2IhFgT4g8CnsweYpSSIQmOl9ujBtw8FswZXGWAOvI4/qEQG0AcKMyOCFPwqWmCQDbEcIeg0wX57DiIdSFexM8LiFXYbBbnxih8oQ1pYhZTwZ/niGV+pb4eSrLimQr91xl8tkIfUy3MiEuEmAKxeYEgIQJiVYgd8rJiQxU24wozWBFDNmJJjDR+c4hj+MIgP7k+VpAmDoxR2Jfm5A3NF9ucIWBHKPD+/Iy4YHl+sFYuRxY/nAt2hS9kxg/p8PPGhw3Nhcf3D5DPHevhC+NjFTofRPl+MfKxOEWUHaWwx0352UFS3hRi57yCWMVYPCEfbki5Pp4myo+Kk8eJF2ZyQqLk8eDLQRhgAX9ABxJYU0EuyASC9r7GPvhP3hMIOEAM0gEf2CuYoRGJsh4hvMaCQvAnRHyQNzzOT9bLBwWQ/zrMyq/2IE3WWyAbkQWeQpwDQkE2/C+RjRIOe0sATyAj+Id3DqxcGG82rNL+f88Psd8ZJmTCFIxkyCNdbciSGED0JwYTA4k2uD7ujXviYfDqC6sTzsDdh+bx3Z7wlNBBeES4Qegk3JkiKBL/FGU46IT6gYpcpP6YC9wSarrgfrgXVIfKuA6uD+xxZ+iHiftAzy6QZSnilmaF/pP232bww2oo7MiOZJQ8guxLtv55pKqtqsuwijTXP+ZHHmvqcL5Zwz0/+2f9kH0ebEN/tsQWYQewc9gp7AJ2FGsEdOwE1oS1YcekeHh3PZHtriFvMbJ4sqCO4B/+hlZWmsk8x1rHXscv8r58/nTpMxqwckUzxIL0jHw6E74R+HS2kOswiu7k6OQMgPT9In98vYmWvTcQnbbv3II/APA6MTg4eOQ7F3ICgH1u8PY//J2zZsBXhzIA5w9zJeICOYdLLwT4lFCDd5oeMAJmwBrOxwm4Ak/gCwJACIgEcSAJTIbRZ8B9LgbTwCwwH5SAMrAcrAbrwSawFewEe8B+0AiOglPgLLgEroAb4B7cPd3gBegH78BnBEFICBWhIXqIMWKB2CFOCAPxRgKQMCQGSUJSkHREiEiQWcgCpAwpR9YjW5AaZB9yGDmFXEA6kDtIF9KLvEY+oRiqgmqhhqglOhploEw0FI1DJ6Hp6FS0EC1Gl6Jr0Wp0N9qAnkIvoTfQTvQFOoABTBnTwUwwe4yBsbBILBlLw8TYHKwUq8CqsTqsGa7zNawT68M+4kSchtNxe7iDg/F4nItPxefgS/D1+E68AW/Fr+FdeD/+jUAlGBDsCB4ENmE8IZ0wjVBCqCBsJxwinIH3UjfhHZFI1CFaEd3gvZhEzCTOJC4hbiDWE08SO4iPiQMkEkmPZEfyIkWSOKR8UglpHWk36QTpKqmb9EFJWclYyUkpUClZSahUpFShtEvpuNJVpWdKn8nqZAuyBzmSzCPPIC8jbyM3ky+Tu8mfKRoUK4oXJY6SSZlPWUupo5yh3Ke8UVZWNlV2V45WFijPU16rvFf5vHKX8kcVTRVbFZbKRBWJylKVHSonVe6ovKFSqZZUX2oyNZ+6lFpDPU19SP2gSlN1UGWr8lTnqlaqNqheVX2pRlazUGOqTVYrVKtQO6B2Wa1Pnaxuqc5S56jPUa9UP6x+S31Ag6YxRiNSI0djicYujQsaPZokTUvNAE2eZrHmVs3Tmo9pGM2MxqJxaQto22hnaN1aRC0rLbZWplaZ1h6tdq1+bU1tZ+0E7enaldrHtDt1MB1LHbZOts4ynf06N3U+jTAcwRzBH7F4RN2IqyPe647U9dXl65bq1uve0P2kR9cL0MvSW6HXqPdAH9e31Y/Wn6a/Uf+Mft9IrZGeI7kjS0fuH3nXADWwNYgxmGmw1aDNYMDQyDDIUGS4zvC0YZ+RjpGvUabRKqPjRr3GNGNvY4HxKuMTxs/p2nQmPZu+lt5K7zcxMAk2kZhsMWk3+WxqZRpvWmRab/rAjGLGMEszW2XWYtZvbmwebj7LvNb8rgXZgmGRYbHG4pzFe0sry0TLhZaNlj1WulZsq0KrWqv71lRrH+up1tXW122INgybLJsNNldsUVsX2wzbStvLdqidq53AboNdxyjCKPdRwlHVo27Zq9gz7Qvsa+27HHQcwhyKHBodXo42H508esXoc6O/Obo4Zjtuc7w3RnNMyJiiMc1jXjvZOnGdKp2uj6WODRw7d2zT2FfOds58543Ot11oLuEuC11aXL66urmKXetce93M3VLcqtxuMbQYUYwljPPuBHc/97nuR90/erh65Hvs9/jL094zy3OXZ884q3H8cdvGPfYy9eJ4bfHq9KZ7p3hv9u70MfHh+FT7PPI18+X5bvd9xrRhZjJ3M1/6OfqJ/Q75vWd5sGazTvpj/kH+pf7tAZoB8QHrAx4GmgamB9YG9ge5BM0MOhlMCA4NXhF8i23I5rJr2P0hbiGzQ1pDVUJjQ9eHPgqzDROHNYej4SHhK8PvR1hECCMaI0EkO3Jl5IMoq6ipUUeiidFR0ZXRT2PGxMyKORdLi50Suyv2XZxf3LK4e/HW8ZL4lgS1hIkJNQnvE/0TyxM7x48eP3v8pST9JEFSUzIpOSF5e/LAhIAJqyd0T3SZWDLx5iSrSdMnXZisPzl78rEpalM4Uw6kEFISU3alfOFEcqo5A6ns1KrUfi6Lu4b7gufLW8Xr5Xvxy/nP0rzSytN60r3SV6b3ZvhkVGT0CViC9YJXmcGZmzLfZ0Vm7cgazE7Mrs9RyknJOSzUFGYJW3ONcqfndojsRCWizqkeU1dP7ReHirfnIXmT8pryteCHfJvEWvKLpKvAu6Cy4MO0hGkHpmtMF05vm2E7Y/GMZ4WBhb/NxGdyZ7bMMpk1f1bXbObsLXOQOalzWuaazS2e2z0vaN7O+ZT5WfN/L3IsKi96uyBxQXOxYfG84se/BP1SW6JaIi65tdBz4aZF+CLBovbFYxevW/ytlFd6scyxrKLsyxLukou/jvl17a+DS9OWti9zXbZxOXG5cPnNFT4rdpZrlBeWP14ZvrJhFX1V6aq3q6esvlDhXLFpDWWNZE3n2rC1TevM1y1f92V9xvoblX6V9VUGVYur3m/gbbi60Xdj3SbDTWWbPm0WbL69JWhLQ7VldcVW4taCrU+3JWw79xvjt5rt+tvLtn/dIdzRuTNmZ2uNW03NLoNdy2rRWklt7+6Ju6/s8d/TVGdft6Vep75sL9gr2ft8X8q+m/tD97ccYByoO2hxsOoQ7VBpA9Iwo6G/MaOxsympqeNwyOGWZs/mQ0ccjuw4anK08pj2sWXHKceLjw+eKDwxcFJ0su9U+qnHLVNa7p0ef/p6a3Rr+5nQM+fPBp49fY557sR5r/NHL3hcOHyRcbHxkuulhjaXtkO/u/x+qN21veGy2+WmK+5XmjvGdRy/6nP11DX/a2evs69fuhFxo+Nm/M3btybe6rzNu91zJ/vOq7sFdz/fm3efcL/0gfqDiocGD6v/sPmjvtO181iXf1fbo9hH9x5zH794kvfkS3fxU+rTimfGz2p6nHqO9gb2Xnk+4Xn3C9GLz30lf2r8WfXS+uXBv3z/ausf39/9Svxq8PWSN3pvdrx1ftsyEDXw8F3Ou8/vSz/ofdj5kfHx3KfET88+T/tC+rL2q83X5m+h3+4P5gwOijhijuxTAIMVTUsD4PUOAKhJANDg+YwyQX7+kxVEfmaVIfCfsPyMKCuuANTB7/foPvh1cwuAvdvg8Qvqq00EIIoKQJw7QMeOHa5DZzXZuVJaiPAcsDn6a2pOKvg3RX7m/CHun1sgVXUGP7f/ApgGfHDsriOAAAAAimVYSWZNTQAqAAAACAAEARoABQAAAAEAAAA+ARsABQAAAAEAAABGASgAAwAAAAEAAgAAh2kABAAAAAEAAABOAAAAAAAAAJAAAAABAAAAkAAAAAEAA5KGAAcAAAASAAAAeKACAAQAAAABAAAAFKADAAQAAAABAAAAGAAAAABBU0NJSQAAAFNjcmVlbnNob3QbxQoxAAAACXBIWXMAABYlAAAWJQFJUiTwAAAB1GlUWHRYTUw6Y29tLmFkb2JlLnhtcAAAAAAAPHg6eG1wbWV0YSB4bWxuczp4PSJhZG9iZTpuczptZXRhLyIgeDp4bXB0az0iWE1QIENvcmUgNi4wLjAiPgogICA8cmRmOlJERiB4bWxuczpyZGY9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMiPgogICAgICA8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0iIgogICAgICAgICAgICB4bWxuczpleGlmPSJodHRwOi8vbnMuYWRvYmUuY29tL2V4aWYvMS4wLyI+CiAgICAgICAgIDxleGlmOlBpeGVsWURpbWVuc2lvbj4yNDwvZXhpZjpQaXhlbFlEaW1lbnNpb24+CiAgICAgICAgIDxleGlmOlBpeGVsWERpbWVuc2lvbj4yMDwvZXhpZjpQaXhlbFhEaW1lbnNpb24+CiAgICAgICAgIDxleGlmOlVzZXJDb21tZW50PlNjcmVlbnNob3Q8L2V4aWY6VXNlckNvbW1lbnQ+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+CjwveDp4bXBtZXRhPgpKZgjLAAAAHGlET1QAAAACAAAAAAAAAAwAAAAoAAAADAAAAAwAAAETEHSxzgAAAN9JREFUSA1ilJNX+s9ARcA4aiDFoTnUw5CRgUHFTIlBXFmMgV+cj+H///8Mbx+9Y3j94A3D0+vPGH59/40RRDi9zC3IxWAdbckgKieMoQkk8O7Ze4Z9cw4x/PzyE0Uep4HOafYMEiriYMUgzU+vPQO7UEJVgkFMQQQs/uHlR4bd0/ahuBSrgTLaUgz28TZgTY8uP2Y4svQEw/9/iAxlGmjEoGapApY/suQYw8NLT8BsEIHVQANPXQZtR02woo2dWxm+vP0K1wBisHKwMgTX+TMwszAx3Dx2h+HMhnNweQAAAAD//wm7Z2wAAAFtSURBVGOUk1f6z4AG7BKsGWS1pBl+ff/NsLp+PZoshOtb5snAJ8LL8PbJO4Ydk/bA1TBiM9A5zZ5BQkWc4c/vvwwv7ryAK0ZmiCmIMbBxsjJ8ePWJYWvPDrgUXgPhqvAwSDLw399/DJd2X8VjHAPD1/dfGR6cfwRXg9WFTil2DJJqEgzfP/9gWNe8Ca6YGAZWA00DjRnULJUZGIDRtapuHcPvn3+IMQusBquB6jaqDCZ+hmAFl3ZdYbi85xqKgdKakgzWkZYMDIwMDA8vPWY4ufo0XB6rgezcbAzexR4MnDwcDP///We4cfQ2w6OLjxmYWJgYRBVEGDTt1BnYudjAhhxbeZLh/tmH+A0EycrrywJdYcHAyAR0Bg7w5PozhkMLj4IthSnB6kKYpLCcEIOJvxGDsIwgAyMjwuCvH78x3DxyG4xBKQEZ4DUQppCVnYWBT5wP6Fomhu8fvjJ8/fgdHGEweWSaKAORNRBiAwBghdPpiOOz5gAAAABJRU5ErkJggg=="]


Yep lets test the regular base64 string without the URI

brianf-aws · 2025-03-17T02:04:06Z

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md

+}
+```
+
+## 2. Create a connector 


Got it, Lets link an example since they would need a secret Arn and a role Arn from my understanding (Using this as a reference https://github.com/opensearch-project/ml-commons/blob/main/docs/tutorials/aws/semantic_search_with_cohere_embedding_model.md#42-create-connector)

mingshl · 2025-03-17T18:32:08Z

address commented in the last commit, can you please review the change and approve? @ohltyler @nathaliellenaa @brianf-aws

Signed-off-by: Mingshi Liu <[email protected]>

brianf-aws

Thanks for addressing my comments approved!

nathaliellenaa

LGTM, thanks @mingshl!

mingshl requested review from b4sjoo, dhrubo-os, jngz-es, model-collapse, rbhavna, ylwu-amzn, zane-neo, Zhangxunmt, austintlee, HenryL27 and xinyual as code owners March 14, 2025 16:29

add standard blueprint for vector search

e0ef6e7

Signed-off-by: Mingshi Liu <[email protected]>

mingshl force-pushed the main-new-blueprints branch from a27e123 to e0ef6e7 Compare March 14, 2025 16:30

mingshl had a problem deploying to ml-commons-cicd-env March 14, 2025 16:32 — with GitHub Actions Error

mingshl temporarily deployed to ml-commons-cicd-env March 14, 2025 16:32 — with GitHub Actions Inactive

mingshl had a problem deploying to ml-commons-cicd-env March 14, 2025 16:32 — with GitHub Actions Failure

mingshl temporarily deployed to ml-commons-cicd-env March 14, 2025 16:32 — with GitHub Actions Inactive

nathaliellenaa reviewed Mar 14, 2025

View reviewed changes

...rence_blueprints/standard_blueprints/bedrock_connector_titan_embedding_standard_blueprint.md Outdated Show resolved Hide resolved

nathaliellenaa reviewed Mar 14, 2025

View reviewed changes

...erence_blueprints/standard_blueprints/cohere_connector_image_embedding_standard_blueprint.md Outdated Show resolved Hide resolved

brianf-aws suggested changes Mar 14, 2025

View reviewed changes

ohltyler reviewed Mar 14, 2025

View reviewed changes

mingshl temporarily deployed to ml-commons-cicd-env March 15, 2025 20:57 — with GitHub Actions Inactive

mingshl requested review from ohltyler, nathaliellenaa and brianf-aws March 15, 2025 20:59

mingshl had a problem deploying to ml-commons-cicd-env March 15, 2025 21:59 — with GitHub Actions Failure

brianf-aws suggested changes Mar 17, 2025

View reviewed changes

remove add trusted endpoint

8703ec4

Signed-off-by: Mingshi Liu <[email protected]>

mingshl force-pushed the main-new-blueprints branch from cccf5f0 to 8703ec4 Compare March 17, 2025 18:35

mingshl had a problem deploying to ml-commons-cicd-env March 17, 2025 18:36 — with GitHub Actions Error

mingshl had a problem deploying to ml-commons-cicd-env March 17, 2025 18:36 — with GitHub Actions Failure

mingshl had a problem deploying to ml-commons-cicd-env March 17, 2025 18:36 — with GitHub Actions Error

use english sentence in predict

bcd4bc1

Signed-off-by: Mingshi Liu <[email protected]>

mingshl temporarily deployed to ml-commons-cicd-env March 17, 2025 19:57 — with GitHub Actions Inactive

mingshl had a problem deploying to ml-commons-cicd-env March 17, 2025 19:57 — with GitHub Actions Error

mingshl had a problem deploying to ml-commons-cicd-env March 17, 2025 19:57 — with GitHub Actions Failure

mingshl temporarily deployed to ml-commons-cicd-env March 17, 2025 19:57 — with GitHub Actions Inactive

brianf-aws approved these changes Mar 17, 2025

View reviewed changes

mingshl had a problem deploying to ml-commons-cicd-env March 17, 2025 21:00 — with GitHub Actions Failure

nathaliellenaa approved these changes Mar 17, 2025

View reviewed changes

Zhangxunmt approved these changes Mar 18, 2025

View reviewed changes

jngz-es approved these changes Mar 18, 2025

View reviewed changes

mingshl merged commit 4e2f31c into opensearch-project:main Mar 18, 2025
7 of 13 checks passed

This was referenced Mar 24, 2025

[Doc] Add new sets of blueprint for embedding model with no pre and post processing functions #3619

Closed

[DOC] An index page to guide users to choose model blueprints when using ml inference processors opensearch-project/documentation-website#9495

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add standard blueprint for vector search #3659

Add standard blueprint for vector search #3659

mingshl commented Mar 14, 2025 •

edited

Loading

brianf-aws left a comment

brianf-aws Mar 14, 2025

mingshl Mar 15, 2025

brianf-aws Mar 17, 2025

mingshl Mar 17, 2025

brianf-aws Mar 14, 2025

mingshl Mar 14, 2025

brianf-aws Mar 17, 2025

brianf-aws Mar 14, 2025

mingshl Mar 15, 2025

brianf-aws Mar 17, 2025

rbhavna Mar 18, 2025

brianf-aws Mar 14, 2025

mingshl Mar 15, 2025

brianf-aws Mar 17, 2025

brianf-aws Mar 14, 2025

mingshl Mar 14, 2025

brianf-aws Mar 17, 2025

ohltyler Mar 14, 2025

mingshl Mar 15, 2025

ohltyler Mar 18, 2025

ohltyler Mar 14, 2025

mingshl Mar 15, 2025

rbhavna Mar 18, 2025

brianf-aws left a comment

brianf-aws Mar 17, 2025

brianf-aws Mar 17, 2025

brianf-aws Mar 17, 2025

brianf-aws Mar 17, 2025

brianf-aws Mar 17, 2025

mingshl commented Mar 17, 2025

brianf-aws left a comment

nathaliellenaa left a comment

		# Model Blueprints for Vector Search

		OpenSearch provides two approaches for implementing embedding models, depending on your needs:


		## 2. Create connector for Amazon Bedrock:

		If you are using self-managed Opensearch, you should supply AWS credentials:

Add standard blueprint for vector search #3659

Add standard blueprint for vector search #3659

Conversation

mingshl commented Mar 14, 2025 • edited Loading

Description

Related Issues

Check List

brianf-aws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brianf-aws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mingshl commented Mar 17, 2025

brianf-aws left a comment

Choose a reason for hiding this comment

nathaliellenaa left a comment

Choose a reason for hiding this comment

mingshl commented Mar 14, 2025 •

edited

Loading