feat(config): let indexed multi-LLM members configure Vertex AI project/region#2384
Open
cdbartholomew wants to merge 1 commit into
Open
feat(config): let indexed multi-LLM members configure Vertex AI project/region#2384cdbartholomew wants to merge 1 commit into
cdbartholomew wants to merge 1 commit into
Conversation
…ct/region
Indexed multi-LLM members previously carried only provider/api_key/model/
base_url, so a 'vertexai' member could not initialize (its client requires a
project id, and the region defaults to us-central1). That made vertexai
unusable as a member of a failover/round-robin chain.
Add optional vertexai_project_id / vertexai_region to LLMMemberConfig, parse
them from {prefix}LLM_{n}_VERTEXAI_PROJECT_ID / _VERTEXAI_REGION (global and
per-op prefixes), thread them through the member build path, and accept them on
LLMProvider so an explicit per-instance value wins while existing single-LLM
setups still fall back to the global config.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extends the indexed multi-LLM layer (#2365) so a
vertexaimember can actually be used in a failover / round-robin chain.Today an indexed member (
HINDSIGHT_API_LLM_<n>_*) carries onlyprovider / api_key / model / base_url. Thevertexaiprovider additionally needs a project id (and a region; it defaults tous-central1) —GeminiLLM._init_vertexairaises"VERTEXAI_PROJECT_ID is required"without one. AndLLMProvider.__init__previously sourced Vertex project/region only from global config viaget_config(), with no way to set them per instance. Net effect:vertexaicould not be a member of a chain — only the global/primary provider.Change
LLMMemberConfig: add optionalvertexai_project_id/vertexai_region._parse_llm_members: read{prefix}LLM_{n}_VERTEXAI_PROJECT_ID/_VERTEXAI_REGION(global and theRETAIN_/REFLECT_/CONSOLIDATION_per-op prefixes), mirroring the existing per-member env keys._member_to_llm: thread the two fields into the member'sLLMConfig.LLMProvider.__init__: acceptvertexai_project_id/vertexai_region; an explicit per-instance value wins, otherwise fall back to global config (llm_vertexai_project_id/llm_vertexai_region→us-central1). Single-LLM setups are unchanged.Why
Without this, a multi-LLM chain can only fail over between key-based providers; a
vertexaimember can't initialize. This lets operators run, e.g., a key-based primary with a Vertex AI failover member (or vice-versa) targeting a specific project/region.Tests
tests/test_multi_llm_config.py(+4):REFLECT_) parses themNone(no regression)uv run pytest tests/test_multi_llm_config.py tests/test_multi_llm_provider.py→ 38 passed.ruff check/ruff format --check/ty checkclean.Notes / non-goals
HINDSIGHT_API_LLM_MAX_RETRIES, per-op overrides) layered on top of the chain.