feat(proxy): add prefixed CLI env var fallbacks#28167
Conversation
…thropic, Bedrock, Vertex) (BerriAI#28133) Squash-merged by litellm-agent from cwang-otto's PR.
Squash-merged by litellm-agent from Cyberfilo's PR.
|
|
Greptile SummaryThis PR adds
Confidence Score: 4/5Safe to merge; changes are purely additive and backward-compatible — existing deployments using the unprefixed env vars continue to work unchanged. The production code change is straightforward and low-risk: extending Click's envvar from a string to a list is a well-understood pattern with no side-effects on existing behavior. The only gap is that the accompanying test validates Click's stored parameter metadata rather than exercising actual env var resolution at runtime, so a Click regression in list-envvar priority would go undetected. The test file could benefit from a runtime resolution test, but neither file requires blocking attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/proxy_cli.py | Adds LITELLM_* prefixed env var aliases for 11 CLI options, keeping existing unprefixed names as fallbacks; fully backward-compatible. |
| tests/test_litellm/proxy/test_proxy_cli.py | New test verifies Click parameter metadata (envvar list and ordering) for all 11 updated options; does not exercise runtime env var resolution via CliRunner. |
Reviews (1): Last reviewed commit: "feat(proxy): add prefixed cli env var fa..." | Re-trigger Greptile
| def test_run_server_prefers_litellm_prefixed_envvars(self): | ||
| from litellm.proxy.proxy_cli import run_server | ||
|
|
||
| expected_envvars = { | ||
| "host": ["LITELLM_HOST", "HOST"], | ||
| "port": ["LITELLM_PORT", "PORT"], | ||
| "num_workers": ["LITELLM_NUM_WORKERS", "NUM_WORKERS"], | ||
| "debug": ["LITELLM_DEBUG", "DEBUG"], | ||
| "detailed_debug": ["LITELLM_DETAILED_DEBUG", "DETAILED_DEBUG"], | ||
| "ssl_keyfile_path": ["LITELLM_SSL_KEYFILE_PATH", "SSL_KEYFILE_PATH"], | ||
| "ssl_certfile_path": ["LITELLM_SSL_CERTFILE_PATH", "SSL_CERTFILE_PATH"], | ||
| "keepalive_timeout": ["LITELLM_KEEPALIVE_TIMEOUT", "KEEPALIVE_TIMEOUT"], | ||
| "timeout_worker_healthcheck": [ | ||
| "LITELLM_TIMEOUT_WORKER_HEALTHCHECK", | ||
| "TIMEOUT_WORKER_HEALTHCHECK", | ||
| ], | ||
| "max_requests_before_restart": [ | ||
| "LITELLM_MAX_REQUESTS_BEFORE_RESTART", | ||
| "MAX_REQUESTS_BEFORE_RESTART", | ||
| ], | ||
| "enforce_prisma_migration_check": [ | ||
| "LITELLM_ENFORCE_PRISMA_MIGRATION_CHECK", | ||
| "ENFORCE_PRISMA_MIGRATION_CHECK", | ||
| ], | ||
| } | ||
|
|
||
| envvars_by_option = { | ||
| param.name: param.envvar for param in run_server.params if param.name | ||
| } | ||
|
|
||
| for option_name, envvars in expected_envvars.items(): | ||
| assert envvars_by_option[option_name] == envvars |
There was a problem hiding this comment.
Test only validates metadata, not runtime resolution priority
The test confirms that each Click parameter stores the expected envvar list in the correct order, but it never actually invokes run_server with conflicting env vars to verify that Click resolves LITELLM_HOST over HOST at runtime. If Click's list-envvar resolution behavior ever regressed (e.g., due to a version change that reversed iteration order), this test would still pass. A complementary test using CliRunner with both env vars set — one to a sentinel value — would give stronger proof of the precedence contract.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
🤖 litellm-agent: Auto-merge skipped — the staging branch Please rebase your branch onto |
|
Rebased this branch onto shin_agent_oss_staging_05_18_2026 and pushed the updated branch. |
7ed4a1f to
3a16720
Compare
PR overviewMedium: Pause toggle does not block model routingThis PR adds an Admin UI pause/resume switch for DB models. The new call sends Security review
Risk: 5/10 |
| if (!accessToken) return; | ||
| try { | ||
| setPausingModelId(modelId); | ||
| await modelPatchUpdateCall(accessToken, { blocked }, modelId); |
There was a problem hiding this comment.
Medium: Pause toggle does not block routing
/model/{model_id}/update parses this as updateDeployment, which only merges model_info, model_name, and litellm_params; the top-level blocked field is ignored while the request still succeeds. A user with an existing key can keep sending requests to a model after an admin clicks Pause, so send and persist the flag through the backend contract and ensure routing rejects blocked deployments before showing success.
Summary
LITELLM_*aliases for the env-backed proxy CLI optionstest_proxy_cli.pyNotes
TIMEOUT_WORKER_HEALTHCHECKwas added after the issue was filed, so this updates the full current set of env-backed proxy flags rather than only the original ten.Testing
uv run --extra proxy --extra extra_proxy pytest tests/test_litellm/proxy/test_proxy_cli.py::TestProxyInitializationHelpers::test_run_server_prefers_litellm_prefixed_envvars -quv run --extra proxy --extra extra_proxy pytest tests/test_litellm/proxy/test_proxy_cli.py -q(32 passed, 1 unrelated failure becausehypercornis not installed in this local env)Closes #27791