feat(models): add @extensible decorator to get_api_key()#1258
Open
Krashnicov wants to merge 1 commit intoagent0ai:developmentfrom
Open
feat(models): add @extensible decorator to get_api_key()#1258Krashnicov wants to merge 1 commit intoagent0ai:developmentfrom
Krashnicov wants to merge 1 commit intoagent0ai:developmentfrom
Conversation
e3b6ea4 to
ea88e18
Compare
Add @extensible decorator to get_api_key() in models.py, enabling plugins to intercept or replace the API key resolution logic for any LLM provider. Currently, get_api_key() resolves API keys exclusively from OS environment variables via os.getenv(). This means external secrets backends (OpenBao/Vault, AWS Secrets Manager, Azure Key Vault, etc.) cannot provide API keys without injecting them into the process environment — a workaround that is fragile and bypasses the existing SecretsManager architecture. With @extensible, a plugin can intercept the start hook to resolve API keys from any backend, falling through to the default dotenv behaviour when no plugin is active. To avoid a circular import (settings.py imports models at module level, and models importing helpers.extension triggers the cycle settings -> models -> extension -> subagents -> plugins -> extension), the top-level `import models` in settings.py is moved inside the three functions that actually use it. This is the standard Python pattern for breaking import cycles. No behavioural change for existing users — the decorator only adds extension hooks around the existing function, and the deferred import in settings.py has no performance impact since Python caches module imports after the first load.
ea88e18 to
f17682d
Compare
Author
Why
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
get_api_key()inmodels.pyresolves API keys exclusively from OS environment variables viaos.getenv(). This means external secrets backends (OpenBao/Vault, AWS Secrets Manager, Azure Key Vault, etc.) cannot provide API keys to LiteLLM without injecting them into the process environment — a workaround that is fragile, doesn't support cache TTLs, and bypasses the existing SecretsManager architecture.While
helpers/secrets.pyfactory functions were recently made extensible (#1246), the actual API key resolution path for chat/embedding/utility models goes throughmodels.get_api_key()→dotenv.get_dotenv_value()→os.getenv(), completely bypassing SecretsManager.Solution
Add the
@extensibledecorator toget_api_key(), enabling plugins to intercept API key resolution at themodels_get_api_key_startextension point.This is a 2-line change:
extensiblefromhelpers.extension@extensibledecorator toget_api_key()How it works
With
@extensible, a plugin extension can:data["result"]in a_starthook to provide an API key from any backend, short-circuiting the default dotenv lookupos.getenv()behaviour when no plugin is activedata["args"]ordata["kwargs"]to alter the service name lookupBackward Compatibility
No behavioural change for existing users. The
@extensibledecorator only adds extension hooks around the existing function. Without any registered extensions,get_api_key()behaves identically to before.Motivating Use Case
OpenBao secrets backend plugin that needs to provide LiteLLM API keys from a KV v2 secrets engine, with TTL-based caching and automatic fallback to
.envfiles.Related
models_get_api_key_start/models_get_api_key_end