Text Generation Inference (TGI) hosting
TGI is Hugging Face's inference server, powering their own Inference Endpoints. It offers continuous batching, quantization support, and guided decoding. Choose it when you deploy through Hugging Face infrastructure or want their supported path; elsewhere the ecosystem has largely consolidated on vLLM.
Providers with a ready-made Text Generation Inference (TGI) path
| Provider | How | |
|---|---|---|
| Hugging Face | Inference Endpoints deploy any Hub model on TGI with autoscaling (the native path) | Deploy |
| RunPod | Run the ghcr.io/huggingface/text-generation-inference image on any pod | Deploy |
| Vast.ai | Run the official TGI Docker image on any instance | Deploy |
Everywhere else: `docker run --gpus all ghcr.io/huggingface/text-generation-inference --model-id <hf-repo>`.
Which GPU to rent
Same weight-size math as vLLM; TGI's default configs are more conservative on KV cache, so it fits slightly tighter on marginal GPUs.
Live floors refreshed 2026-08-23| Model class | GPU | VRAM | From $/hr | Cheapest at | |
|---|---|---|---|---|---|
| Up to ~17B (FP8) / ~34B (INT4) | RTX 4090 | 24 GB | $0.134marketplace | Vast.ai | Rent |
| Up to ~23B (FP8) / ~46B (INT4) | RTX 5090 | 32 GB | $0.321marketplace | Vast.ai | Rent |
| Up to ~34B (FP8) / ~69B (INT4) | L40S | 48 GB | $0.467marketplace | Vast.ai | Rent |
| Up to ~58B (FP8) / ~116B (INT4) | H100 SXM | 80 GB | $1.34marketplace | Vast.ai | Rent |
| Up to ~102B (FP8) / ~205B (INT4) | H200 | 141 GB | $2.00spot | Verda | |
| Up to ~130B (FP8) / ~261B (INT4) — MoE giants go multi-GPU | B200 | 180 GB | $3.06spot | Verda |
Rent vs API: when self-hosting Text Generation Inference (TGI) pays off
Self-hosting wins when utilization is high and steady. A rented GPU bills every hour whether you use it or not, while APIs bill per token. Run your numbers in the breakeven calculator, or check any open model's page (e.g. Llama 3.3 70B) for a side-by-side estimate.