A GPU can cost less on paper and still be the wrong purchase. Compare the bill first, then check whether the rented system can serve your requests at the latency and quality you need.
1. Compare the same job
Record the model/version, input and output tokens, context lengths, concurrency, acceptable latency, region and quality requirements. If the hosted API and self-hosted model differ, test answer quality before comparing costs. Use input and output rates from one provider and one service tier, not the cheapest half of two different offers.
For ordinary uncached text requests:
API cost = input_tokens / 1,000,000 × input_rate
+ output_tokens / 1,000,000 × output_rate
Rental cost = GPU_count × price_per_GPU_hour × billable_hours
+ storage + network + other_operating_costs
Use actual billable token categories for caching, batch, tools or other modalities. Add any fixed API fees. If an offer quotes the whole machine rather than each GPU, do not multiply it by GPU count again.
Billable hours include paid setup and idle time, not only time spent generating tokens. A month is not always 730 hours; use your billing period. Storage may continue after compute stops: Runpod’s Pod pricing distinguishes running and stopped storage charges. Vast’s pricing documentation also lists storage and bandwidth separately.
2. Work one example
These are invented planning inputs, not current vendor prices or measured performance. All amounts are USD, before tax.
| Input | Example |
|---|---|
| Monthly input tokens | 100 million |
| Monthly output tokens | 20 million |
| API input / million | $0.50 |
| API output / million | $2.00 |
| One GPU, hourly rental | $0.80 |
| Billable rental hours | 730 |
| Storage allowance | $20 |
| Network allowance | $0 |
| Operator time and other costs | $100 |
API: 100 × 0.50 + 20 × 2.00 = $90.
Rental: 1 × 0.80 × 730 + 20 + 0 + 100 = $704.
At this workload, the API is $614 cheaper. The zero network allowance is an example assumption, not a promise about any provider.
3. Solve for a cost crossover
Hold the input/output ratio at five input tokens for every output token. Each million output tokens then carries five million input tokens:
API cost per million output, including its inputs = 5 × 0.50 + 2.00 = $4.50
Cost crossover = 704 / 4.50 = 156.44 million output tokens per month
Corresponding input volume = 782.22 million tokens per month
This is a cost-only crossover, not a validated self-hosting threshold. It assumes rental cost remains $704 as usage grows. If serving that workload needs another GPU, more hours, a replica or extra storage, recalculate. If costs vary with traffic, include those variable costs rather than treating them as fixed.
Copy this into a JavaScript console to substitute your own assumptions:
const inputM = 100, outputM = 20;
const inputRate = 0.50, outputRate = 2.00;
const rental = 1 * 0.80 * 730 + 20 + 0 + 100;
const api = inputM * inputRate + outputM * outputRate;
const ratio = inputM / outputM;
const crossoverOutputM = rental / (ratio * inputRate + outputRate);
console.log({ api, rental, crossoverOutputM });
4. Check capacity before switching
The example crossover requires about 59.53 output tokens per second averaged across all 730 hours: 156.44 × 1,000,000 / (730 × 3,600). That is a required average, not a GPU benchmark. It excludes peaks and cannot establish whether any particular hardware can do the job.
Run a representative test with the exact weights, quantization, serving engine, context lengths and concurrency you plan to use. Record completed requests, aggregate output throughput, time to first token, per-request generation speed, errors and peak memory. Test the busiest interval, not just a monthly average. Include input processing and recovery time in the test.
Do not substitute a vendor’s maximum throughput figure for your workload. A configuration that meets the average but misses your latency target has not passed. Keep a fallback plan and its cost in the comparison.
Decision
Use the API when its complete bill is lower or a rental has not passed your capacity test. Consider rental only when the matched workload fits, meets your service targets, and leaves enough savings to justify operating it. There is no universal monthly token count at which everyone should switch.
Start with the model listings and GPU listings, then apply this worksheet to one real offer. The rental checklist covers what to verify before paying.
Provider documentation checked November 17, 2026. The arithmetic above is illustrative and reproducible; no hardware benchmark was performed.