Skip to content

feat(vertexai): support meta's llama models#17

Draft
CorieW wants to merge 11 commits into
mainfrom
vertexai-support-llama
Draft

feat(vertexai): support meta's llama models#17
CorieW wants to merge 11 commits into
mainfrom
vertexai-support-llama

Conversation

@CorieW

@CorieW CorieW commented Apr 28, 2026

Copy link
Copy Markdown
Member

Support Meta's Llama models in VertexAI.

Testing

image

Todo: Evaluating options for testing this, as requires deploying infrastructure.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for Meta (Llama) models within the Vertex AI plugin by implementing an OpenAI-compatible API client and the necessary request/response mapping logic. Key changes include the addition of the VertexAiMetaOptions class for model configuration, the meta method for model referencing, and updates to the plugin's discovery logic to include Meta models. Review feedback identifies critical syntax errors in map literals where an invalid ?variable syntax was used instead of collection if statements, as well as a potential null-safety issue when joining text parts.

Comment on lines +48 to +57
'version': ?version,
'temperature': ?temperature,
'topP': ?topP,
'maxTokens': ?maxTokens,
'stop': ?stop,
'presencePenalty': ?presencePenalty,
'frequencyPenalty': ?frequencyPenalty,
'seed': ?seed,
'user': ?user,
'llamaGuard': ?llamaGuard,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The ?variable syntax used in this map literal is not valid Dart. To conditionally include entries in a map based on whether a value is null, you should use the collection if syntax.

Suggested change
'version': ?version,
'temperature': ?temperature,
'topP': ?topP,
'maxTokens': ?maxTokens,
'stop': ?stop,
'presencePenalty': ?presencePenalty,
'frequencyPenalty': ?frequencyPenalty,
'seed': ?seed,
'user': ?user,
'llamaGuard': ?llamaGuard,
if (version != null) 'version': version,
if (temperature != null) 'temperature': temperature,
if (topP != null) 'topP': topP,
if (maxTokens != null) 'maxTokens': maxTokens,
if (stop != null) 'stop': stop,
if (presencePenalty != null) 'presencePenalty': presencePenalty,
if (frequencyPenalty != null) 'frequencyPenalty': frequencyPenalty,
if (seed != null) 'seed': seed,
if (user != null) 'user': user,
if (llamaGuard != null) 'llamaGuard': llamaGuard,

Comment on lines +133 to +140
'temperature': ?options.temperature,
'top_p': ?options.topP,
'max_tokens': ?options.maxTokens,
'stop': ?options.stop,
'presence_penalty': ?options.presencePenalty,
'frequency_penalty': ?options.frequencyPenalty,
'seed': ?options.seed,
'user': ?options.user,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The ?variable syntax used in this map literal is not valid Dart. To conditionally include entries in a map based on whether a value is null, you should use the collection if syntax.

Suggested change
'temperature': ?options.temperature,
'top_p': ?options.topP,
'max_tokens': ?options.maxTokens,
'stop': ?options.stop,
'presence_penalty': ?options.presencePenalty,
'frequency_penalty': ?options.frequencyPenalty,
'seed': ?options.seed,
'user': ?options.user,
if (options.temperature != null) 'temperature': options.temperature,
if (options.topP != null) 'top_p': options.topP,
if (options.maxTokens != null) 'max_tokens': options.maxTokens,
if (options.stop != null) 'stop': options.stop,
if (options.presencePenalty != null) 'presence_penalty': options.presencePenalty,
if (options.frequencyPenalty != null) 'frequency_penalty': options.frequencyPenalty,
if (options.seed != null) 'seed': options.seed,
if (options.user != null) 'user': options.user,


Object _toMetaContent(List<Part> parts) {
if (parts.every((part) => part.isText)) {
return parts.map((part) => part.text).join('\n');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While parts.every((part) => part.isText) ensures all parts are text parts, part.text on the base Part class typically returns a nullable String?. Using join on an iterable of nullable strings can result in the literal string "null" being included in the output if any value is null. It is safer to cast the parts to TextPart to ensure non-nullability.

Suggested change
return parts.map((part) => part.text).join('\n');
return parts.cast<TextPart>().map((part) => part.text).join('\n');

@CorieW
CorieW requested a review from cabljac April 28, 2026 12:35
@CorieW
CorieW force-pushed the vertexai-support-llama branch from b35337f to 4db5379 Compare May 11, 2026 12:25
@CorieW
CorieW force-pushed the vertexai-support-llama branch 2 times, most recently from b6cdbbf to 4d20853 Compare May 11, 2026 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant