🚀 Inference Gateway Examples

Examples for deploying, configuring, and using the Inference Gateway across different environments and use cases.

---

⚠️ Early Development Notice: This project is in its early stages of development. Some deployment examples may not work as expected or are currently in draft mode. We are actively working on completing and testing all examples using design-first principles. Please be patient as we continue to improve the documentation and examples.

Note: The examples in each individual project repository are more accurate and likely to be working. This monolithic examples repository is still a work in progress (WIP). For the most reliable examples, please check the individual project repositories.

If you encounter issues, feel free to open an issue or contribute to help us improve.

Monitoring Examples

Choose your preferred deployment method:

Docker (Recommended for Development)

cd quickstart/docker
task

Kubernetes (Recommended for Production)

cd quickstart/kubernetes
task setup  # Interactive setup with automatic cluster creation

Don't have a cluster? Use task setup for interactive cluster creation with k3d.

From main examples directory? Use task quickstart:k8s as a shortcut.

📁 Repository Structure

examples/
├── quickstart/ # Get started quickly
│ ├── docker/ # Docker deployment
│ └── kubernetes/ # Kubernetes deployment
├── sdks/ # SDK usage examples
│ ├── go/ # Go SDK examples
│ ├── python/ # Python SDK examples
│ ├── typescript/ # TypeScript SDK examples
│ └── rust/ # Rust SDK examples
├── a2a/ # Agent-to-Agent examples
│ ├── getting-started/ # Basic A2A setup
│ ├── google-calendar/ # Calendar agent example
│ └── custom-agents/ # Custom agent examples
├── mcp/ # Model Context Protocol examples
│ ├── servers/ # MCP server implementations
│ └── clients/ # MCP client examples
├── monitoring/ # Observability examples
│ ├── prometheus/ # Metrics collection
│ ├── grafana/ # Dashboards
│ └── tracing/ # Distributed tracing
├── security/ # Security configurations
│ ├── authentication/ # Auth setups
│ ├── tls/ # TLS/SSL examples
│ └── rbac/ # Role-based access
└── tutorials/ # Step-by-step guides
├── first-request/ # Your first API call
├── model-routing/ # Advanced routing
├── rate-limiting/ # Rate limiting setup
└── custom-providers/ # Adding new providers

🎯 Choose Your Path

I want to...

Get started quickly

→ Go to quickstart/ for Docker or Kubernetes deployment

Use SDKs in my application

→ Check sdks/ for language-specific examples

Create AI agents

→ Visit a2a/ for Agent-to-Agent examples

Integrate with MCP

→ See mcp/ for Model Context Protocol examples

Deploy to production

→ Review deployment/ for advanced deployment scenarios

Monitor and observe

→ Check monitoring/ for observability setups

Secure my deployment

→ Explore security/ for security configurations

Learn step by step

→ Follow tutorials/ for guided learning

🛠 Prerequisites

General Requirements

• Task: Install Task for running examples
• Git: For cloning repositories
• curl: For testing API endpoints

Deployment-Specific

• Docker: For container-based examples
• kubectl: For Kubernetes examples
• Node.js: For UI and TypeScript examples
• Go: For Go SDK examples
• Python: For Python SDK examples
• Rust: For Rust SDK examples

API Keys

You'll need at least one API key from supported providers:

• OpenAI (OPENAI_API_KEY)
• Anthropic (ANTHROPIC_API_KEY)
• Groq (GROQ_API_KEY)
• DeepSeek (DEEPSEEK_API_KEY)
• Cohere (COHERE_API_KEY)
• or local Ollama (OLLAMA_API_URL)

📖 Example Categories

🚀 Quickstart Examples

Perfect for getting up and running immediately:

• Docker: Single command deployment
• Kubernetes: Production-ready cluster deployment

💻 SDK Examples

Language-specific client implementations:

• Go: Native Go applications
• Python: Data science and AI workflows
• TypeScript: Web applications and Node.js services
• Rust: High-performance applications

🤖 A2A Examples

Agent-to-Agent protocol implementations:

• Getting Started: Basic agent setup
• Google Calendar: Real-world integration
• Custom Agents: Build your own agents

🔌 MCP Examples

Model Context Protocol integrations:

• Servers: Custom MCP server implementations
• Clients: MCP client integrations

🚀 Deployment Examples

Production deployment scenarios:

• Helm: Kubernetes package management
• Terraform: Infrastructure as Code
• Docker Compose: Multi-service orchestration
• Cloud: AWS, GCP, Azure specific deployments

Monitoring Examples

Observability and monitoring:

• Prometheus: Metrics collection
• Grafana: Visualization dashboards
• Tracing: Request tracing setup

🔒 Security Examples

Security and authentication:

• Authentication: User authentication setups
• TLS: SSL/TLS configuration
• RBAC: Role-based access control

📚 Tutorial Examples

Guided learning experiences:

• First Request: Making your first API call
• Model Routing: Advanced request routing
• Rate Limiting: Implementing rate limits
• Custom Providers: Adding new AI providers

🌟 Getting Started

Method 1: Quick Navigation (Recommended)

# From the main examples directory
task quickstart:k8s    # Kubernetes with interactive setup
task quickstart:docker # Docker deployment

Method 2: Direct Navigation

1. Choose your deployment method:

   # For development
   cd quickstart/docker && task
   
   # For production
   cd quickstart/kubernetes && task setup

2. Test your deployment:

   # Health check
   curl http://localhost:8080/health
   
   # List models
   curl http://localhost:8080/v1/models

3. Explore examples:

   # Try SDK examples
   cd sdks/python && task
   
   # Build a UI
   cd ui/nextjs && task
   
   # Create agents
   cd a2a/getting-started && task

🤝 Contributing

Found an issue or want to add an example? We welcome contributions!

1. Fork the repository
2. Create a feature branch
3. Add your example with proper documentation
4. Follow the existing structure and Taskfile patterns
5. Submit a pull request

Example Guidelines

• Use Taskfile.yml for automation (no shell scripts)
• Include comprehensive README.md
• Add .env.example for configuration
• Follow language-specific best practices
• Include error handling and troubleshooting

📖 Additional Resources

• Documentation - Official documentation
• API Reference - API documentation
• GitHub - Source code
• Discord - Community support

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.