vLLM hosting
vLLM is the most widely deployed open-source inference engine, built around PagedAttention for high-throughput serving. It exposes an OpenAI-compatible API, supports tensor parallelism across multiple GPUs, continuous batching, and quantized checkpoints (FP8, GPTQ, AWQ). If you are serving an open model to real traffic and don't have a reason to pick something else, you pick vLLM.
Providers with a ready-made vLLM path
| Provider | How | |
|---|---|---|
| RunPod | Official vLLM template in RunPod's template gallery, plus vLLM workers on Serverless | Deploy |
| Vast.ai | vLLM template in Vast's recommended templates; set the model via env var | Deploy |
| Novita AI | vLLM instance template in the template marketplace | Deploy |
Everywhere else: Any provider that rents raw GPU VMs runs vLLM via the official Docker image: `docker run --gpus all vllm/vllm-openai --model <hf-repo>`.
Which GPU to rent
Rule of thumb: model weights need ~1.1 GB VRAM per billion params at FP8, ~0.55 at INT4/AWQ, plus 20-40% headroom for KV cache depending on context length and batch size.
Live floors refreshed 2026-08-23| Model class | GPU | VRAM | From $/hr | Cheapest at | |
|---|---|---|---|---|---|
| Up to ~17B (FP8) / ~34B (INT4) | RTX 4090 | 24 GB | $0.134marketplace | Vast.ai | Rent |
| Up to ~23B (FP8) / ~46B (INT4) | RTX 5090 | 32 GB | $0.321marketplace | Vast.ai | Rent |
| Up to ~34B (FP8) / ~69B (INT4) | L40S | 48 GB | $0.467marketplace | Vast.ai | Rent |
| Up to ~58B (FP8) / ~116B (INT4) | H100 SXM | 80 GB | $1.34marketplace | Vast.ai | Rent |
| Up to ~102B (FP8) / ~205B (INT4) | H200 | 141 GB | $2.00spot | Verda | |
| Up to ~130B (FP8) / ~261B (INT4) — MoE giants go multi-GPU | B200 | 180 GB | $3.06spot | Verda |
Rent vs API: when self-hosting vLLM pays off
Self-hosting wins when utilization is high and steady. A rented GPU bills every hour whether you use it or not, while APIs bill per token. Run your numbers in the breakeven calculator, or check any open model's page (e.g. Llama 3.3 70B) for a side-by-side estimate.