Note

No longer active in the web and mobile apps. The code lives in the archived _taro directory, but the architecture deserves a write-up anyway.

The plan was to serve TaroTarot on Lightning Studio, with schedulers on AWS Lambda and models pulled from Hugging Face repos. Lightning doesn't need Docker at all; it's built to keep you away from cloud plumbing so you can spend time on models. My intent was to move off Lightning once the short-term goals were hit, hence the containers.

The shape of it ​

Frontend → FastAPI app (:8080) → Ollama server (:11434) → local models from Hugging Face / S3

  • Both containers sit on one bridge network, tarotarot-network, and find each other by service name instead of IP.
  • app runs python:3.12-slim as a non-root user (UID 1000) with uvicorn taro.app:app.
  • ollama runs ollama/ollama:latest with a start_ollama.sh that preloads the models.
  • A shared ollama-data volume holds model weights so neither container re-downloads on restart.
bash
docker-compose up -d          # start
docker-compose logs -f app    # watch the API
docker-compose down -v        # stop and wipe models

Why I moved to Vertex AI anyway ​

Self-hosting gives full control over model versions, no per-token billing, and data that never leaves my infra. It also means I own uptime, GPUs that cost money while idle, and a Compose file that doesn't auto-scale.

For a product with a bounded lifespan and moderate traffic, a hosted API is pragmatically enough. So for now TaroTarot serves through Google Vertex AI, and the Compose stack waits for the day Tom needs that sovereignty.

Improvements I never got to: an ngrok container, and swapping FastAPI for Lightning's own serving.

Full notes, including the complete Compose file and pros/cons tables, on GitBook.