Skip to content

Latest commit

 

History

History
68 lines (47 loc) · 1.67 KB

README.md

File metadata and controls

68 lines (47 loc) · 1.67 KB

PyRemote

Remote security-first Python code execution with a simple frontend UI powered by Kubernetes

k8s Startup

  1. Build the images. From the api directory build the images using the following two commands
docker build . -f fastapi_app/Dockerfile -t etweisberg/fastapi-app:latest
docker build . -f celery_worker/Dockerfile -t etweisberg/celery-worker:latest
  1. Create a kubernetes cluster
kind create cluster --name cis1912
  1. Load images into cluster
kind load docker-image etweisberg/celery-worker:latest --name cis1912
kind load docker-image etweisberg/fastapi-app:latest --name cis1912
  1. Create the namespace first
kubectl apply -f k8s/namespace.yaml
  1. Apply all other k8s deployments and services
kubectl apply -f k8s/
  1. Port-forward to test out the API locally
kubectl port-forward svc/fastapi-service 8000:8000 -n api-apps

Set Up

  1. Celery task run as a k8s deployment

    • Takes a string of code to execute + list of dependencies to install
    • Sets up virtual env in temp directory
    • Installs dependencies
    • Executes code in virtual environment
    • Cleans up temp directory after execution
    • Store job output in AWS S3
    • Implement resource constraints + timeouts on Celery task

  2. FastAPI for submitting code

    • has endpoint for accepting code and requirements
    • processes code and requirements to ensure security (TBD) before submitting Celery task
    • uses k8s deployment
    • expose FastAPI with Ingress + Ingress Controller (NGINX)

  3. Simple frontend in Next.js deployed with Vercel for simplicity

    • Code editor window
    • STDOUT + STDERR displays
    • Run button (sends request to FastAPI)