Skip to content

What is the correct way to do multimodal embeddings on an image? #1511

@twointum

Description

@twointum

I have code that works:

model = MultiModalEmbeddingModel.from_pretrained("multimodalembedding@001")

  try:
      image_object = Image.load_from_file("temp_image.png")
      embeddings = model.get_embeddings(image=image_object)

  except Exception as e:
      print(f"Error embedding image from temp_image.png: {e}")
      return None, None

  #this is the image vector
  image_embedding = embeddings.image_embedding

but I get this:

/workspaces/vscode-python-devcontainer/my_project/.venv/lib/python3.12/site-packages/vertexai/_model_garden/_model_garden_models.py:278: UserWarning: This feature is deprecated as of June 24, 2025 and will be removed on June 24, 2026. For details, see https://cloud.google.com/vertex-ai/generative-ai/docs/deprecations/genai-vertexai-sdk.
  warning_logs.show_deprecation_warning()
/workspaces/vscode-python-devcontainer/my_project/.venv/lib/python3.12/site-packages/vertexai/vision_models/_vision_models.py:153: UserWarning: This feature is deprecated as of June 24, 2025 and will be removed on June 24, 2026. For details, see https://cloud.google.com/vertex-ai/generative-ai/docs/deprecations/genai-vertexai-sdk.
  warning_logs.show_deprecation_warning()

That link has a section for text embeddings:
https://cloud.google.com/vertex-ai/generative-ai/docs/deprecations/genai-vertexai-sdk#embeddings

But the code for embed_content says it's text only!

def embed_content(

Am I not using the correct libraries? Thank you!

Metadata

Metadata

Assignees

Labels

priority: p3Desirable enhancement or fix. May not be included in next release.status:awaiting user responsetype: questionRequest for information or clarification. Not an issue.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions