-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
Feature Request: Support litellm / OpenRouter for Embedding Models
Problem
Currently, OpenViking's embedding configuration only supports three providers:
volcengine(Doubao)openai(text-embedding-3-large, etc.)jina
The VLM layer supports litellm which can bridge to OpenRouter, Ollama, Anthropic, DeepSeek, Gemini, and many other providers. However, the embedding layer does not support litellm, forcing users to either:
- Use Volcengine Doubao (requires separate account/API)
- Use OpenAI embeddings (separate API key + cost)
- Use Jina (free, but limited model selection)
This is a significant blocker for users who want to centralize all model calls through OpenRouter for its simplicity.
Use Case
Many developers prefer OpenRouter due to its simplicity — unified API, single dashboard, single billing for dozens of providers. OpenViking's VLM already supports litellm → OpenRouter, but the embedding layer is a hard gap that breaks the unified workflow.
Proposed Solution
Extend the embedding provider list to include litellm, similar to how VLM already works:
"embedding": {
"dense": {
"provider": "litellm",
"model": "openai/text-embedding-3-small",
"api_base": "https://openrouter.ai/api/v1",
"api_key": "<openrouter-key>",
"dimension": 1024
}
}This would allow:
- OpenRouter embeddings
- Ollama local embeddings (via litellm bridge)
- vLLM served embeddings
- Any OpenAI-compatible embedding endpoint
Why litellm?
litellm already handles OpenAI-compatible embedding endpoints cleanly. Since VLM already uses litellm successfully, the infrastructure is already in place. Extending it to embedding would be a natural addition.
Alternatives Considered
- Jina embeddings — Works today, but model selection is limited and requires outbound API call to jina.ai
- OpenAI embeddings — Works but adds cost and separate API dependency
- Volcengine Doubao — Not free, requires separate account
None of these match the flexibility of litellm for users with existing OpenRouter/Ollama setups.
Additional Context
- This would bring embedding parity with how VLM already works via litellm
- Many AI agents are standardizing on OpenRouter for all LLM/embedding needs
- Embedding model format is standardized (OpenAI-compatible), so litellm should work with minimal changes
Metadata
Metadata
Assignees
Labels
Type
Projects
Status