feat(vertexai): support meta's llama models#17
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces support for Meta (Llama) models within the Vertex AI plugin by implementing an OpenAI-compatible API client and the necessary request/response mapping logic. Key changes include the addition of the VertexAiMetaOptions class for model configuration, the meta method for model referencing, and updates to the plugin's discovery logic to include Meta models. Review feedback identifies critical syntax errors in map literals where an invalid ?variable syntax was used instead of collection if statements, as well as a potential null-safety issue when joining text parts.
| 'version': ?version, | ||
| 'temperature': ?temperature, | ||
| 'topP': ?topP, | ||
| 'maxTokens': ?maxTokens, | ||
| 'stop': ?stop, | ||
| 'presencePenalty': ?presencePenalty, | ||
| 'frequencyPenalty': ?frequencyPenalty, | ||
| 'seed': ?seed, | ||
| 'user': ?user, | ||
| 'llamaGuard': ?llamaGuard, |
There was a problem hiding this comment.
The ?variable syntax used in this map literal is not valid Dart. To conditionally include entries in a map based on whether a value is null, you should use the collection if syntax.
| 'version': ?version, | |
| 'temperature': ?temperature, | |
| 'topP': ?topP, | |
| 'maxTokens': ?maxTokens, | |
| 'stop': ?stop, | |
| 'presencePenalty': ?presencePenalty, | |
| 'frequencyPenalty': ?frequencyPenalty, | |
| 'seed': ?seed, | |
| 'user': ?user, | |
| 'llamaGuard': ?llamaGuard, | |
| if (version != null) 'version': version, | |
| if (temperature != null) 'temperature': temperature, | |
| if (topP != null) 'topP': topP, | |
| if (maxTokens != null) 'maxTokens': maxTokens, | |
| if (stop != null) 'stop': stop, | |
| if (presencePenalty != null) 'presencePenalty': presencePenalty, | |
| if (frequencyPenalty != null) 'frequencyPenalty': frequencyPenalty, | |
| if (seed != null) 'seed': seed, | |
| if (user != null) 'user': user, | |
| if (llamaGuard != null) 'llamaGuard': llamaGuard, |
| 'temperature': ?options.temperature, | ||
| 'top_p': ?options.topP, | ||
| 'max_tokens': ?options.maxTokens, | ||
| 'stop': ?options.stop, | ||
| 'presence_penalty': ?options.presencePenalty, | ||
| 'frequency_penalty': ?options.frequencyPenalty, | ||
| 'seed': ?options.seed, | ||
| 'user': ?options.user, |
There was a problem hiding this comment.
The ?variable syntax used in this map literal is not valid Dart. To conditionally include entries in a map based on whether a value is null, you should use the collection if syntax.
| 'temperature': ?options.temperature, | |
| 'top_p': ?options.topP, | |
| 'max_tokens': ?options.maxTokens, | |
| 'stop': ?options.stop, | |
| 'presence_penalty': ?options.presencePenalty, | |
| 'frequency_penalty': ?options.frequencyPenalty, | |
| 'seed': ?options.seed, | |
| 'user': ?options.user, | |
| if (options.temperature != null) 'temperature': options.temperature, | |
| if (options.topP != null) 'top_p': options.topP, | |
| if (options.maxTokens != null) 'max_tokens': options.maxTokens, | |
| if (options.stop != null) 'stop': options.stop, | |
| if (options.presencePenalty != null) 'presence_penalty': options.presencePenalty, | |
| if (options.frequencyPenalty != null) 'frequency_penalty': options.frequencyPenalty, | |
| if (options.seed != null) 'seed': options.seed, | |
| if (options.user != null) 'user': options.user, |
|
|
||
| Object _toMetaContent(List<Part> parts) { | ||
| if (parts.every((part) => part.isText)) { | ||
| return parts.map((part) => part.text).join('\n'); |
There was a problem hiding this comment.
While parts.every((part) => part.isText) ensures all parts are text parts, part.text on the base Part class typically returns a nullable String?. Using join on an iterable of nullable strings can result in the literal string "null" being included in the output if any value is null. It is safer to cast the parts to TextPart to ensure non-nullability.
| return parts.map((part) => part.text).join('\n'); | |
| return parts.cast<TextPart>().map((part) => part.text).join('\n'); |
b35337f to
4db5379
Compare
…d add corresponding test
…unsupported roles and part types
…rove error handling for ToolResponse.ref
b6cdbbf to
4d20853
Compare
Support Meta's Llama models in VertexAI.
Testing
Todo: Evaluating options for testing this, as requires deploying infrastructure.