Ollama hosting
Ollama wraps llama.cpp with a one-line install, a model registry, and an OpenAI-compatible API. It runs GGUF quantized models efficiently on a single GPU (or CPU), making it ideal for development, internal tools, and low-traffic apps. It is not built for high-concurrency production serving. That's vLLM/SGLang territory.
Providers with a ready-made Ollama path
| Provider | How | |
|---|---|---|
| RunPod | Ollama template in the gallery; expose port 11434 | Deploy |
| Vast.ai | Ollama template in recommended templates | Deploy |
| Novita AI | Ollama template in the template marketplace | Deploy |
Everywhere else: `curl -fsSL https://ollama.com/install.sh | sh && ollama run llama3.3` on any GPU VM.
Which GPU to rent
GGUF Q4 quants need roughly 0.6 GB VRAM per billion params. A 24 GB card (RTX 4090) comfortably runs 32B-class models at Q4.
Live floors refreshed 2026-08-23| Model class | GPU | VRAM | From $/hr | Cheapest at | |
|---|---|---|---|---|---|
| Up to ~17B (FP8) / ~34B (INT4) | RTX 4090 | 24 GB | $0.134marketplace | Vast.ai | Rent |
| Up to ~23B (FP8) / ~46B (INT4) | RTX 5090 | 32 GB | $0.321marketplace | Vast.ai | Rent |
| Up to ~34B (FP8) / ~69B (INT4) | L40S | 48 GB | $0.467marketplace | Vast.ai | Rent |
| Up to ~58B (FP8) / ~116B (INT4) | H100 SXM | 80 GB | $1.34marketplace | Vast.ai | Rent |
| Up to ~102B (FP8) / ~205B (INT4) | H200 | 141 GB | $2.00spot | Verda | |
| Up to ~130B (FP8) / ~261B (INT4) — MoE giants go multi-GPU | B200 | 180 GB | $3.06spot | Verda |
Rent vs API: when self-hosting Ollama pays off
Self-hosting wins when utilization is high and steady. A rented GPU bills every hour whether you use it or not, while APIs bill per token. Run your numbers in the breakeven calculator, or check any open model's page (e.g. Llama 3.3 70B) for a side-by-side estimate.