FastAPI Template for LLMs

Introduction

• FastAPI Template for fast building LLMs API using FastAPI and Langchain
• Support:
  • FastAPI application
  • Celery worker
  • PostgreSQL
  • pgAdmin
  • RabbitMQ
  • Redis

Note: This template is not ready for production.

Documentation

• FastAPI

Installation

• Docker
• Docker Compose

Requirements

Environment variables: copy from .env.example to .env and change the values.

cp .env.example .env

Docker

You can run the following command to start the server:

docker-compose up -d

Development

Development environment requires:

• Python >= 3.9
• Redis >= 6
• PostgreSQL >= 13
• Poetry

To start development environment, run the following command:

cd src
poetry install
# Please active Poetry virtual environment before running the following command
poetry shell

# Init database
alembic upgrade head

# Start celery worker
celery -A celeryApp.worker worker -l info -Q main-queue -c 1

Note: Windows require gevent to run celery worker.

celery -A celeryApp.worker worker -l info -Q main-queue -c 1 -P gevent

python app/main.py

If you get stuck at Loading segmented_maxsim_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)..., remove the cache:

rm -rf /home/{user}/.cache/torch_extensions/py310_cu117

Gitlab CI/CD

Please add these variables to Gitlab CI/CD:

• CI_JOB_TOKEN: Gitlab CI/CD job token
• CI_PROJECT_NAME: Gitlab project name
• CI_REGISTRY: Gitlab registry
• DOCKER_IMAGE_BACKEND: Docker image name for backend (eg: registry.gitlab.com/username/project_name/backend)
• ENV_FILE_DEV (File): Environment variables file for development (eg: .env.dev)
• GIT_REPO: Current git repository
• SSH_PRIVATE_KEY: SSH private key that added to deployment server
• SSH_SERVER: Deployment server
• SSH_USER: Deployment user
• STACK_NAME: Docker stack name

With .gitlab-ci.yml, please change services.command to your registry server.

---
services:
  - name: docker:dind
    alias: docker
    # Change insecure-registry to your registry
    command: ["--insecure-registry=<container registry server>"]

Endpoints

Endpoint	Method	Description
/	GET	Hello world
/api/v1/docs		Swagger UI

TODO

• Traefik
• Testing
• OAuth2

Contributing

License

• LICENSE