Skip to content

feat: add litellm template#372

Merged
Marvin-Cypher merged 1 commit into
Phala-Network:mainfrom
phala-agent:feat/template-litellm
May 28, 2026
Merged

feat: add litellm template#372
Marvin-Cypher merged 1 commit into
Phala-Network:mainfrom
phala-agent:feat/template-litellm

Conversation

@phala-agent
Copy link
Copy Markdown
Contributor

Summary

  • Add the litellm prebuilt template for the upstream BerriAI/litellm repository.
  • Run the official LiteLLM Proxy image in CPU-safe smoke mode with an empty model_list, no provider credentials, no GPU, no model downloads, and no database requirement.
  • Add upstream-sourced icon templates/icons/litellm.jpg from ui/litellm-dashboard/public/assets/logos/litellm_logo.jpg.

Template files

  • templates/prebuilt/litellm/docker-compose.yml
  • templates/prebuilt/litellm/README.md
  • templates/icons/litellm.jpg
  • templates/config.json

Environment variables

  • LITELLM_IMAGE_TAG (optional, default main-stable)
  • LITELLM_LOG (optional, default INFO)
  • LITELLM_MASTER_KEY (optional placeholder; set a strong value before exposing real model routes)
  • OPENAI_API_KEY (optional placeholder for user-added model routes)
  • ANTHROPIC_API_KEY (optional placeholder for user-added model routes)
  • GOOGLE_API_KEY (optional placeholder for user-added model routes)

No real secrets are included.

Validation

  • python3 templates/validate.py
  • git diff --check origin/main...HEAD
  • docker compose -f templates/prebuilt/litellm/docker-compose.yml config >/dev/null
  • Static audit ✅: README coverage, unique config ID, icon file present, correct Phala prebuilt repo route, no env_file, no host bind mounts, no privileged/host networking, no literal secret patterns.

Phala Cloud deployment smoke

  • Profile/workspace: hermes-admin-cvm-check / h4x's projects
  • Deploy command used tdx.small, --wait, --no-public-logs, and --no-public-sysinfo.
  • Smoke CVM: fb7f8f74-eb9f-455f-9794-4b976469d916
  • App ID: 3eab5fdf4c5df64bcdfad72da59098df3d8a9a45
  • phala ps showed /dstack-litellm-1 running and healthy.
  • Endpoint probes:
    • GET /health/liveliness -> HTTP 200, body "I'm alive!"
    • GET /health/readiness -> HTTP 200, body includes "status":"healthy"
    • GET /v1/models -> HTTP 200, body {"data":[],"object":"list"}

Cleanup

  • Temporary CVM delete command returned success.
  • Follow-up lookup returned The requested CVM was not found, so cleanup is verified.

cc @Marvin-Cypher for review.

@phala-agent
Copy link
Copy Markdown
Contributor Author

cc @Marvin-Cypher for review.

@Marvin-Cypher Marvin-Cypher force-pushed the feat/template-litellm branch from 346bf8e to f0a73cd Compare May 28, 2026 02:24
Copy link
Copy Markdown
Contributor

@Marvin-Cypher Marvin-Cypher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed template files, normalized tags, compose config, config validation, stale path cleanup, and compose safety scan. Checks are passing.

@Marvin-Cypher Marvin-Cypher merged commit f99db2b into Phala-Network:main May 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants