Skip to content

feat(models): add @extensible decorator to get_api_key()#1258

Open
Krashnicov wants to merge 1 commit intoagent0ai:developmentfrom
Deimos-AI:pr/extensible-models-get-api-key
Open

feat(models): add @extensible decorator to get_api_key()#1258
Krashnicov wants to merge 1 commit intoagent0ai:developmentfrom
Deimos-AI:pr/extensible-models-get-api-key

Conversation

@Krashnicov
Copy link

Problem

get_api_key() in models.py resolves API keys exclusively from OS environment variables via os.getenv(). This means external secrets backends (OpenBao/Vault, AWS Secrets Manager, Azure Key Vault, etc.) cannot provide API keys to LiteLLM without injecting them into the process environment — a workaround that is fragile, doesn't support cache TTLs, and bypasses the existing SecretsManager architecture.

While helpers/secrets.py factory functions were recently made extensible (#1246), the actual API key resolution path for chat/embedding/utility models goes through models.get_api_key()dotenv.get_dotenv_value()os.getenv(), completely bypassing SecretsManager.

Solution

Add the @extensible decorator to get_api_key(), enabling plugins to intercept API key resolution at the models_get_api_key_start extension point.

This is a 2-line change:

  1. Import extensible from helpers.extension
  2. Add @extensible decorator to get_api_key()

How it works

With @extensible, a plugin extension can:

  • Set data["result"] in a _start hook to provide an API key from any backend, short-circuiting the default dotenv lookup
  • Fall through to the default os.getenv() behaviour when no plugin is active
  • Modify data["args"] or data["kwargs"] to alter the service name lookup

Backward Compatibility

No behavioural change for existing users. The @extensible decorator only adds extension hooks around the existing function. Without any registered extensions, get_api_key() behaves identically to before.

Motivating Use Case

OpenBao secrets backend plugin that needs to provide LiteLLM API keys from a KV v2 secrets engine, with TTL-based caching and automatic fallback to .env files.

Related

@Krashnicov Krashnicov force-pushed the pr/extensible-models-get-api-key branch from e3b6ea4 to ea88e18 Compare March 14, 2026 09:18
Add @extensible decorator to get_api_key() in models.py, enabling
plugins to intercept or replace the API key resolution logic for
any LLM provider.

Currently, get_api_key() resolves API keys exclusively from OS
environment variables via os.getenv(). This means external secrets
backends (OpenBao/Vault, AWS Secrets Manager, Azure Key Vault, etc.)
cannot provide API keys without injecting them into the process
environment — a workaround that is fragile and bypasses the existing
SecretsManager architecture.

With @extensible, a plugin can intercept the start hook to resolve
API keys from any backend, falling through to the default dotenv
behaviour when no plugin is active.

To avoid a circular import (settings.py imports models at module
level, and models importing helpers.extension triggers the cycle
settings -> models -> extension -> subagents -> plugins -> extension),
the top-level `import models` in settings.py is moved inside the
three functions that actually use it. This is the standard Python
pattern for breaking import cycles.

No behavioural change for existing users — the decorator only adds
extension hooks around the existing function, and the deferred
import in settings.py has no performance impact since Python caches
module imports after the first load.
@Krashnicov Krashnicov force-pushed the pr/extensible-models-get-api-key branch from ea88e18 to f17682d Compare March 15, 2026 08:57
@Krashnicov
Copy link
Author

Why helpers/settings.py is included

models.py is imported at module level by settings.py (line 9: import models). Adding from helpers.extension import extensible to models.py creates a circular import:

settings.py → import models → helpers.extension → helpers.subagents
  → helpers.plugins → extension (not yet initialized) → AttributeError

The fix moves import models from top-level in settings.py into the three functions that actually use it: convert_out(), _get_api_key_field(), and _load_sensitive_settings(). This is standard Python circular-import resolution — settings.py was the unusual importer (top-level), while models.py is widely imported across the codebase and shouldn't have import restrictions.

Python caches modules after first load, so the deferred import has zero performance impact on subsequent calls. No behavioural change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant