Skip to content

feat: add litellm template#372

Open
phala-agent wants to merge 1 commit into
Phala-Network:mainfrom
phala-agent:feat/template-litellm
Open

feat: add litellm template#372
phala-agent wants to merge 1 commit into
Phala-Network:mainfrom
phala-agent:feat/template-litellm

Conversation

@phala-agent
Copy link
Copy Markdown
Contributor

Summary

  • Add the litellm prebuilt template for the upstream BerriAI/litellm repository.
  • Run the official LiteLLM Proxy image in CPU-safe smoke mode with an empty model_list, no provider credentials, no GPU, no model downloads, and no database requirement.
  • Add upstream-sourced icon templates/icons/litellm.jpg from ui/litellm-dashboard/public/assets/logos/litellm_logo.jpg.

Template files

  • templates/prebuilt/litellm/docker-compose.yml
  • templates/prebuilt/litellm/README.md
  • templates/icons/litellm.jpg
  • templates/config.json

Environment variables

  • LITELLM_IMAGE_TAG (optional, default main-stable)
  • LITELLM_LOG (optional, default INFO)
  • LITELLM_MASTER_KEY (optional placeholder; set a strong value before exposing real model routes)
  • OPENAI_API_KEY (optional placeholder for user-added model routes)
  • ANTHROPIC_API_KEY (optional placeholder for user-added model routes)
  • GOOGLE_API_KEY (optional placeholder for user-added model routes)

No real secrets are included.

Validation

  • python3 templates/validate.py
  • git diff --check origin/main...HEAD
  • docker compose -f templates/prebuilt/litellm/docker-compose.yml config >/dev/null
  • Static audit ✅: README coverage, unique config ID, icon file present, correct Phala prebuilt repo route, no env_file, no host bind mounts, no privileged/host networking, no literal secret patterns.

Phala Cloud deployment smoke

  • Profile/workspace: hermes-admin-cvm-check / h4x's projects
  • Deploy command used tdx.small, --wait, --no-public-logs, and --no-public-sysinfo.
  • Smoke CVM: fb7f8f74-eb9f-455f-9794-4b976469d916
  • App ID: 3eab5fdf4c5df64bcdfad72da59098df3d8a9a45
  • phala ps showed /dstack-litellm-1 running and healthy.
  • Endpoint probes:
    • GET /health/liveliness -> HTTP 200, body "I'm alive!"
    • GET /health/readiness -> HTTP 200, body includes "status":"healthy"
    • GET /v1/models -> HTTP 200, body {"data":[],"object":"list"}

Cleanup

  • Temporary CVM delete command returned success.
  • Follow-up lookup returned The requested CVM was not found, so cleanup is verified.

cc @Marvin-Cypher for review.

@phala-agent
Copy link
Copy Markdown
Contributor Author

cc @Marvin-Cypher for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant