The tensor core problem nobody mentions in the model card
Nemotron 3, Qwen3.6, and GLM-5.2 all shipped native NVFP4 checkpoints this cycle, calibrated by the model vendor rather than bolted on with a third party quantizer afterward. That matters for quality: a first-party FP4 calibration tends to hold up better on eval than a GPTQ pass done by whoever hosts the weights. But the checkpoint format is only half the story. NVFP4 with microscaling needs Blackwell’s fifth generation tensor cores to execute at native 4 bit throughput, and Hopper (H100, H200) and Ada (L40S, 6000 Ada) don’t have that instruction path. Load an NVFP4 checkpoint on an H100 and the runtime either upcasts to FP8/BF16 for the matmuls or falls back to a slower emulated kernel. Either way you keep the smaller memory footprint and lose the compute win that’s supposed to justify the quant in the first place.
So the question isn’t “should I quantize to FP4.” It’s “which GPU generation turns that quant into cheaper tokens.” Right now that’s B200, B300, RTX 5090, and RTX PRO 6000. Everything else on the fleet, rent it for FP8 or leave the quant question alone.
Sizing the checkpoint before picking the card
Use the same rough footprint math the site’s own sizing guidance uses: about 0.55 GB per billion params at 4 bit, plus 25% headroom for KV cache. Treat this as illustrative. NVFP4’s microscaling adds a small overhead versus a naive INT4 pack, but it’s close enough to rank cards.
Nemotron 3 Nano at 30B total params comes out to roughly 16-17 GB at FP4, call it 21 GB with headroom. That’s a single RTX 5090 job with room to spare, and the 5090 rents almost entirely through community and marketplace tiers where the current discount versus the secure tier runs 70%+ as of this writing, check the RTX 5090 page for today’s spread. Qwen3.6 35B A3B lands in the same bracket, around 25 GB with headroom.
Nemotron 3 Super at 120B total roughly triples that footprint to the mid 80s GB, past what a 5090 can hold and squarely in RTX PRO 6000 or single-B200 territory. GLM-5.2 at 744B total pushes the FP4 footprint past 500 GB with headroom, and no single Blackwell card touches that, so you’re sharding across a multi GPU B200 or B300 node regardless of quant. The quant changes how many nodes you need, not whether you need more than one.
None of this replaces checking the live spec sheet. The math above tells you the footprint; the GPU pages tell you what fits on a given card today.
The B300 anomaly
B300 is the newer, higher spec Blackwell part. The naive assumption is that it always prices above B200. It doesn’t, not consistently. On the current listings, B300 spot capacity sometimes clears below B200 for the same hour, a supply quirk from providers still ramping B300 fleets and discounting to fill them, not a spec regression. So if you’re pricing a Blackwell FP4 deployment, don’t default to B200 because it’s the “established” SKU. Pull both B200 and B300 live and take whichever is cheaper that day; the compute path is close enough between the two that the price, not the label, should decide.
What FP8 on Hopper still wins
If your fleet is H100 or H200 committed capacity you already paid for, don’t chase the NVFP4 checkpoint just because it exists. You’ll pay the VRAM savings back in extra dequant overhead on every forward pass, and depending on batch size that overhead can erase most of the memory win. Serve FP8 on Hopper. Serve NVFP4 on Blackwell. Mixing the two, an FP4 checkpoint served on Hopper silicon, is the trap: it looks like the same “cheap quant” decision on paper, and it’s the one combination in this whole matrix that doesn’t pay off. vLLM and SGLang both gate their FP4 kernels to Blackwell compute capability for exactly this reason, so check the vLLM or SGLang hosting notes before you assume the runtime will do something clever for you on older hardware.
The rule
Compute the model’s FP4 footprint with the 0.55 GB per billion param estimate plus 25% headroom, then check whether it fits on a single Blackwell card on the GPU comparison page. If it fits on an RTX 5090 or RTX PRO 6000, rent that card and run the native NVFP4 checkpoint; the community tier discount on those SKUs plus the compute win stacks into the cheapest per-token number on the board. If it doesn’t fit on a single card, price B200 against B300 same day rather than assuming the newer part costs more. And if your fleet is Hopper or Ada and can’t be reprovisioned this quarter, stay on FP8 and don’t let an NVFP4 release note talk you into a format your tensor cores can’t execute. What this can’t tell you is how much quality your specific eval set loses going from FP8 to FP4 on that vendor’s calibration. That’s a number you get from running your own suite against both checkpoints, not from a spec sheet.