Skip to content

Feat/helm argocd improvements clean #15734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+224 −2
Open

Feat/helm argocd improvements clean #15734

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6e72768
feat(helm): add external secrets operator support
busla Sep 23, 2025
7e53dc4
Merge branch 'BerriAI:main' into feat/helm-argocd-improvements-clean
busla Oct 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 53 additions & 2 deletions deploy/charts/litellm-helm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,9 @@ If `db.useStackgresOperator` is used (not yet implemented):
| `proxyConfigMap.create` | When `true`, render a ConfigMap from `.Values.proxy_config` and mount it. | `true` |
| `proxyConfigMap.name` | When `create=false`, name of the existing ConfigMap to mount. | `""` |
| `proxyConfigMap.key` | Key in the ConfigMap that contains the proxy config file. | `"config.yaml"` |
| `proxy_config.*` | See [values.yaml](./values.yaml) for default settings. Rendered into the ConfigMap’s `config.yaml` only when `proxyConfigMap.create=true`. See [example_config_yaml](../../../litellm/proxy/example_config_yaml/) for configuration examples. | `N/A` |
| `extraContainers[]` | An array of additional containers to be deployed as sidecars alongside the LiteLLM Proxy.
| `proxy_config.*` | See [values.yaml](./values.yaml) for default settings. Rendered into the ConfigMap's `config.yaml` only when `proxyConfigMap.create=true`. See [example_config_yaml](../../../litellm/proxy/example_config_yaml/) for configuration examples. | `N/A` |
| `extraContainers[]` | An array of additional containers to be deployed as sidecars alongside the LiteLLM Proxy. | `[]` |
| `extraResources[]` | An array of additional Kubernetes resources to deploy alongside LiteLLM. Useful for external-secrets, custom secrets, etc. | `[]` |
| `pdb.enabled` | Enable a PodDisruptionBudget for the LiteLLM proxy Deployment | `false` |
| `pdb.minAvailable` | Minimum number/percentage of pods that must be available during **voluntary** disruptions (choose **one** of minAvailable/maxUnavailable) | `null` |
| `pdb.maxUnavailable` | Maximum number/percentage of pods that can be unavailable during **voluntary** disruptions (choose **one** of minAvailable/maxUnavailable) | `null` |
Expand Down Expand Up @@ -149,6 +150,56 @@ data:

Source: [GitHub Gist from troyharvey](https://gist.github.com/troyharvey/4506472732157221e04c6b15e3b3f094)

## Using External Secrets Operator

The chart supports deploying additional Kubernetes resources via the `extraResources` array in values.yaml. This is particularly useful for integrating with the [External Secrets Operator](https://external-secrets.io/) to manage secrets from external systems like AWS Systems Manager Parameter Store, HashiCorp Vault, Azure Key Vault, etc.

### Example: AWS SSM Parameter Store Integration

See [examples/external-secrets-ssm.yaml](./examples/external-secrets-ssm.yaml) for a complete example of using AWS SSM Parameter Store with external-secrets.

Prerequisites:
1. Install external-secrets operator in your cluster
2. Configure IRSA (IAM Role for Service Accounts) for AWS SSM access
3. Store your secrets in SSM Parameter Store

```bash
# Install external-secrets operator
helm repo add external-secrets https://charts.external-secrets.io
helm install external-secrets external-secrets/external-secrets \
-n external-secrets-system --create-namespace

# Deploy LiteLLM with external-secrets
helm install litellm . -f examples/external-secrets-ssm.yaml
```

The example configuration will:
- Create a SecretStore for AWS SSM Parameter Store
- Fetch secrets from SSM parameters like `/litellm/master-key`, `/litellm/openai/api-key`
- Make them available as Kubernetes secrets
- Configure LiteLLM to use these secrets via environment variables

### Custom External Resources

You can also use `extraResources` to deploy any custom Kubernetes resources:

```yaml
extraResources:
- apiVersion: v1
kind: Secret
metadata:
name: my-custom-secret
data:
key: dmFsdWU= # base64 encoded "value"
- apiVersion: v1
kind: ConfigMap
metadata:
name: my-config
data:
config.yaml: |
key: value
```

### Migration Job Settings

The migration job supports both ArgoCD and Helm hooks to ensure database migrations run at the appropriate time during deployments.
Expand Down
134 changes: 134 additions & 0 deletions deploy/charts/litellm-helm/examples/external-secrets-ssm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Example configuration for using external-secrets with AWS SSM Parameter Store
#
# Prerequisites:
# 1. Install external-secrets operator: helm repo add external-secrets https://charts.external-secrets.io && helm install external-secrets external-secrets/external-secrets -n external-secrets-system --create-namespace
# 2. Configure IRSA for the litellm service account or provide AWS credentials
# 3. Store secrets in SSM Parameter Store under the /litellm/ prefix
#
# Usage: helm install litellm . -f examples/external-secrets-ssm.yaml

# Enable service account for IRSA
serviceAccount:
create: true
name: litellm
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/litellm-ssm-role

# Use external secret for master key instead of auto-generated one
masterkeySecretName: litellm-master-key-external

# Example SSM parameters you should create:
# aws ssm put-parameter --name "/litellm/master-key" --value "your-secure-master-key" --type "SecureString"
# aws ssm put-parameter --name "/litellm/db/username" --value "litellm_user" --type "String"
# aws ssm put-parameter --name "/litellm/db/password" --value "secure-db-password" --type "SecureString"
# aws ssm put-parameter --name "/litellm/openai/api-key" --value "sk-..." --type "SecureString"

extraResources:
# SecretStore for AWS SSM Parameter Store
- apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: litellm-ssm-store
spec:
provider:
aws:
service: ParameterStore
region: us-east-1
auth:
serviceAccountRef:
name: litellm

# External secret for LiteLLM master key
- apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: litellm-master-key-external
spec:
refreshInterval: 1h
secretStoreRef:
name: litellm-ssm-store
kind: SecretStore
target:
name: litellm-master-key-external
creationPolicy: Owner
data:
- secretKey: masterkey
remoteRef:
key: /litellm/master-key

# External secret for database credentials (if using external DB)
- apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: litellm-db-external
spec:
refreshInterval: 1h
secretStoreRef:
name: litellm-ssm-store
kind: SecretStore
target:
name: litellm-db-external
creationPolicy: Owner
data:
- secretKey: username
remoteRef:
key: /litellm/db/username
- secretKey: password
remoteRef:
key: /litellm/db/password

# External secret for API keys and other environment variables
- apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: litellm-api-keys
spec:
refreshInterval: 15m
secretStoreRef:
name: litellm-ssm-store
kind: SecretStore
target:
name: litellm-api-keys
creationPolicy: Owner
data:
- secretKey: OPENAI_API_KEY
remoteRef:
key: /litellm/openai/api-key
- secretKey: ANTHROPIC_API_KEY
remoteRef:
key: /litellm/anthropic/api-key
- secretKey: AZURE_API_KEY
remoteRef:
key: /litellm/azure/api-key

# Use the external database secret (uncomment if using external DB)
# db:
# useExisting: true
# secret:
# name: litellm-db-external
# usernameKey: username
# passwordKey: password

# Include the API keys secret as environment variables
environmentSecrets:
- litellm-api-keys

# Update proxy configuration to use environment variables from SSM
proxy_config:
model_list:
- model_name: gpt-4
litellm_params:
model: gpt-4
api_key: os.environ/OPENAI_API_KEY
- model_name: claude-3-sonnet
litellm_params:
model: anthropic/claude-3-sonnet-20240229
api_key: os.environ/ANTHROPIC_API_KEY
- model_name: azure-gpt-4
litellm_params:
model: azure/gpt-4
api_key: os.environ/AZURE_API_KEY
api_base: https://your-resource.openai.azure.com/
api_version: "2024-02-15-preview"
general_settings:
master_key: os.environ/PROXY_MASTER_KEY
4 changes: 4 additions & 0 deletions deploy/charts/litellm-helm/templates/extra-resources.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{{- range .Values.extraResources }}
---
{{ toYaml . }}
{{- end }}
33 changes: 33 additions & 0 deletions deploy/charts/litellm-helm/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,3 +248,36 @@ pdb:
maxUnavailable: null # e.g. 1 or "20%"
annotations: {}
labels: {}

# Additional Kubernetes resources to deploy alongside LiteLLM
# Useful for external-secrets, custom secrets, etc.
extraResources: []
# Example: External Secrets for AWS SSM Parameter Store
# - apiVersion: external-secrets.io/v1beta1
# kind: SecretStore
# metadata:
# name: litellm-ssm-store
# spec:
# provider:
# aws:
# service: ParameterStore
# region: us-east-1
# auth:
# serviceAccountRef:
# name: litellm
# - apiVersion: external-secrets.io/v1beta1
# kind: ExternalSecret
# metadata:
# name: litellm-master-key
# spec:
# refreshInterval: 1h
# secretStoreRef:
# name: litellm-ssm-store
# kind: SecretStore
# target:
# name: litellm-master-key
# creationPolicy: Owner
# data:
# - secretKey: masterkey
# remoteRef:
# key: /litellm/master-key
Loading