Skip to content

feat(proxy): add prefixed CLI env var fallbacks#28167

Open
pntech20 wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
pntech20:feat/prefix-proxy-cli-envvars
Open

feat(proxy): add prefixed CLI env var fallbacks#28167
pntech20 wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
pntech20:feat/prefix-proxy-cli-envvars

Conversation

@pntech20
Copy link
Copy Markdown

Summary

  • add LITELLM_* aliases for the env-backed proxy CLI options
  • keep the existing unprefixed names as fallbacks so current deployments keep working
  • cover the env var ordering in test_proxy_cli.py

Notes

TIMEOUT_WORKER_HEALTHCHECK was added after the issue was filed, so this updates the full current set of env-backed proxy flags rather than only the original ten.

Testing

  • uv run --extra proxy --extra extra_proxy pytest tests/test_litellm/proxy/test_proxy_cli.py::TestProxyInitializationHelpers::test_run_server_prefers_litellm_prefixed_envvars -q
  • uv run --extra proxy --extra extra_proxy pytest tests/test_litellm/proxy/test_proxy_cli.py -q (32 passed, 1 unrelated failure because hypercorn is not installed in this local env)

Closes #27791

cwang-otto and others added 2 commits May 18, 2026 01:35
…thropic, Bedrock, Vertex) (BerriAI#28133)

Squash-merged by litellm-agent from cwang-otto's PR.
Squash-merged by litellm-agent from Cyberfilo's PR.
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented May 18, 2026

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ Cyberfilo
✅ cwang-otto
❌ pntech20
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 18, 2026

Greptile Summary

This PR adds LITELLM_*-prefixed environment variable aliases for all 11 env-backed proxy_cli.py CLI options, keeping the existing unprefixed names as ordered fallbacks so current deployments are unaffected.

  • proxy_cli.py: Each targeted @click.option has its envvar changed from a plain string to a two-element list [\"LITELLM_FOO\", \"FOO\"], giving the prefixed form priority while preserving backward compatibility.
  • test_proxy_cli.py: A new test, test_run_server_prefers_litellm_prefixed_envvars, introspects Click's parameter metadata to assert the exact envvar list (including order) for all 11 updated options.

Confidence Score: 4/5

Safe to merge; changes are purely additive and backward-compatible — existing deployments using the unprefixed env vars continue to work unchanged.

The production code change is straightforward and low-risk: extending Click's envvar from a string to a list is a well-understood pattern with no side-effects on existing behavior. The only gap is that the accompanying test validates Click's stored parameter metadata rather than exercising actual env var resolution at runtime, so a Click regression in list-envvar priority would go undetected.

The test file could benefit from a runtime resolution test, but neither file requires blocking attention.

Important Files Changed

Filename Overview
litellm/proxy/proxy_cli.py Adds LITELLM_* prefixed env var aliases for 11 CLI options, keeping existing unprefixed names as fallbacks; fully backward-compatible.
tests/test_litellm/proxy/test_proxy_cli.py New test verifies Click parameter metadata (envvar list and ordering) for all 11 updated options; does not exercise runtime env var resolution via CliRunner.

Reviews (1): Last reviewed commit: "feat(proxy): add prefixed cli env var fa..." | Re-trigger Greptile

Comment on lines +135 to +166
def test_run_server_prefers_litellm_prefixed_envvars(self):
from litellm.proxy.proxy_cli import run_server

expected_envvars = {
"host": ["LITELLM_HOST", "HOST"],
"port": ["LITELLM_PORT", "PORT"],
"num_workers": ["LITELLM_NUM_WORKERS", "NUM_WORKERS"],
"debug": ["LITELLM_DEBUG", "DEBUG"],
"detailed_debug": ["LITELLM_DETAILED_DEBUG", "DETAILED_DEBUG"],
"ssl_keyfile_path": ["LITELLM_SSL_KEYFILE_PATH", "SSL_KEYFILE_PATH"],
"ssl_certfile_path": ["LITELLM_SSL_CERTFILE_PATH", "SSL_CERTFILE_PATH"],
"keepalive_timeout": ["LITELLM_KEEPALIVE_TIMEOUT", "KEEPALIVE_TIMEOUT"],
"timeout_worker_healthcheck": [
"LITELLM_TIMEOUT_WORKER_HEALTHCHECK",
"TIMEOUT_WORKER_HEALTHCHECK",
],
"max_requests_before_restart": [
"LITELLM_MAX_REQUESTS_BEFORE_RESTART",
"MAX_REQUESTS_BEFORE_RESTART",
],
"enforce_prisma_migration_check": [
"LITELLM_ENFORCE_PRISMA_MIGRATION_CHECK",
"ENFORCE_PRISMA_MIGRATION_CHECK",
],
}

envvars_by_option = {
param.name: param.envvar for param in run_server.params if param.name
}

for option_name, envvars in expected_envvars.items():
assert envvars_by_option[option_name] == envvars
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Test only validates metadata, not runtime resolution priority

The test confirms that each Click parameter stores the expected envvar list in the correct order, but it never actually invokes run_server with conflicting env vars to verify that Click resolves LITELLM_HOST over HOST at runtime. If Click's list-envvar resolution behavior ever regressed (e.g., due to a version change that reversed iteration order), this test would still pass. A complementary test using CliRunner with both env vars set — one to a sentinel value — would give stronger proof of the precedence contract.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@oss-pr-review-agent-shin
Copy link
Copy Markdown
Contributor

🤖 litellm-agent: Auto-merge skipped — the staging branch shin_agent_oss_staging_05_18_2026 has 2 commit(s) not in your branch. Merging as-is would produce a confusing diff on the staging PR.

Please rebase your branch onto shin_agent_oss_staging_05_18_2026 and push; the agent will re-review automatically.

@pntech20
Copy link
Copy Markdown
Author

Rebased this branch onto shin_agent_oss_staging_05_18_2026 and pushed the updated branch.

@pntech20 pntech20 force-pushed the feat/prefix-proxy-cli-envvars branch from 7ed4a1f to 3a16720 Compare May 19, 2026 00:41
@veria-ai
Copy link
Copy Markdown
Contributor

veria-ai Bot commented May 19, 2026

PR overview

Medium: Pause toggle does not block model routing

This PR adds an Admin UI pause/resume switch for DB models. The new call sends blocked in a shape the existing PATCH endpoint ignores, so the dashboard reports success without making the model unavailable.

Security review

  • 1 new security issue(s) were flagged in the latest review.
  • 1 issue(s) remain open on this pull request.

Risk: 5/10

if (!accessToken) return;
try {
setPausingModelId(modelId);
await modelPatchUpdateCall(accessToken, { blocked }, modelId);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: Pause toggle does not block routing

/model/{model_id}/update parses this as updateDeployment, which only merges model_info, model_name, and litellm_params; the top-level blocked field is ignored while the request still succeeds. A user with an existing key can keep sending requests to a model after an admin clicks Pause, so send and persist the flag through the backend contract and ensure routing rejects blocked deployments before showing success.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Scope proxy_cli.py Click envvars under the LITELLM_* prefix used elsewhere in the codebase

4 participants