Skip to content

README.md

Latest commit

 

History

History
39 lines (30 loc) · 916 Bytes

README.md

File metadata and controls

39 lines (30 loc) · 916 Bytes

GPU Services

This directory contains Docker Compose configuration for GPU-accelerated AI services used by Mycelia, including Whisper transcription and Ollama LLM inference.

Services

Whisper ASR

  • Image: onerahmet/openai-whisper-asr-webservice:v1.9.1-gpu
  • Purpose: Audio transcription using OpenAI Whisper

Ollama

  • Image: ollama/ollama:0.13.5
  • Purpose: Local LLM inference with GPU acceleration

Proxy Server

  • Purpose: API gateway providing:
    • OpenAI-compatible transcription endpoint
    • Authentication
    • Unified API access to both services

Prerequisites

  • Docker with GPU support (NVIDIA Docker runtime)
  • NVIDIA GPU with CUDA support

Setup

Tested with

NVIDIA GeForce RTX 4090
CUDA 13.0
Driver Version: 580.95.05

# on your GPU machine
git clone https://github.com/mycelia-tech/mycelia.git
cd mycelia/gpu
docker compose up -d --build