Feature Request: Hybrid AI Provider Settings (Global ENVs + Tenant Overrides) with EU Gateway Support (e.g., Requesty)

[worldwidehigh]

Feature Title

Hybrid Provider Config (System ENVs with Tenant/Workspace UI Overrides) & EU-Compliant Gateway Integration

Describe the feature

I would like to propose a hybrid configuration model for AI Providers (LLMs, Embeddings, Rerankers, Vision) combined with better support for EU-compliant gateways.

This feature consists of two main pillars:

1. The Hybrid Configuration Model

• Global Defaults (via ENV variables): System-wide default API keys, base URLs, models, embeddings, rerankers, and vision models are configured during deployment using ENV variables. This keeps local dev and simple self-hosted setups zero-ops.
• Tenant/Workspace Overrides (via Admin UI): Administrators should have the ability to override these settings on a per-tenant or per-workspace basis inside the UI. Admins can enter tenant-specific API keys, custom endpoints (like regional Azure OpenAI endpoints), and choose specific models/rerankers.

2. Native Support for EU-Compliant Gateways (e.g., Requesty)

For many enterprises, data residency and GDPR compliance are strict requirements. We need the ability to easily integrate European-hosted AI gateways/routers like Requesty.

• Since gateways like Requesty provide an OpenAI-compatible API, we should ensure the application allows fully customizable Base URLs alongside the API keys—both in the global ENV variables (e.g., pointing to https://router.requesty.ai or their Frankfurt gateway) and in the Tenant UI override panel.

Motivation

This hybrid approach solves several critical real-world deployment challenges:

1. GDPR & EU Market Readiness (Why Requesty matters):
   In Europe, hosting data outside the EU or sending sensitive text to US-based endpoints is often a blocker. Gateways like Requesty solve this by hosting their routing layer in Frankfurt, offering GDPR compliance, and never storing request/response bodies. Making it extremely easy to plug in these European gateways by allowing customizable base URLs (via ENVs and UI) will make edgequake highly attractive for European enterprise clients.

2. Enterprise Flexibility & BYOK ("Bring Your Own Key"):
   While system-wide defaults are great for getting started, larger corporate tenants often insist on using their own dedicated AI subscriptions (e.g., their own Azure OpenAI endpoints or local Mistral deployments) for cost-tracking and compliance reasons. Allowing tenant-level overrides is essential for any production-grade multi-tenant RAG system.

3. Infrastructural Security:
   Admins can keep the global system keys securely hidden in the environment variables, while still giving specific tenants the flexibility to safely configure their own endpoints through the UI.

Alternatives considered

• ENV-only Configuration: Restricts multi-tenancy and prevents us from selling edgequake as a SaaS where clients want to use their own models.
• DB-only UI Configuration: Makes automated GitOps deployments harder and increases security risks if system-wide API keys have to be saved in the database without strong hardware-level encryption.

Additional context

No response