My wall of highlights on deploying AI/ML/Data Science stacks in dev and prod environments.

My playground setup

The platforms I use the most:

• Ollama
  • push/pull models for inference
  • containerize models / agents
• Lightning AI Studio
  • containerize model training and quick trials
  • sandboxing (alternatively, a Hugging Face Space)
• AWS EC2 Reserved
  • VPC & terminal local access
• Codespaces
  • frontend dev (used interchangeably with Lightning AI Studio when a project needs a hybrid setup)

Memorial wall

• Always use Docker.
• Ollama serves models faster than other servers, e.g. llama.cpp. Unsure why.

More on the Ollama side in serving LLMs with Ollama.