Sizing GLM-5.2 across GPUs
GLM-5.2 carries 744B total parameters and 40B active per token. At INT4, weights run roughly 0.55GB per billion params, so the base weights alone come to about 410GB. Add the usual 25% headroom for KV cache and you’re provisioning something close to 510GB of GPU memory before a single token gets served. Nothing on the GPU pricing page holds that much memory on one card, including the highest-capacity data center parts like H200 and B200. You’re sizing a cluster, not a single card. The moment you cross from one card to several, a second cost shows up that has nothing to do with per-GPU dollars: how those cards talk to each other.
What tensor parallelism synchronizes
Tensor parallelism splits each layer’s matrix multiply across GPUs and reassembles the result with an all-reduce after every layer. That reassembly is the whole cost of scaling this way. It’s not extra floating-point work, it’s synchronization traffic that every card has to receive before the next layer can start, on every forward pass, for the life of the deployment.
Inside a single NVLink switch domain, commonly 8 GPUs per node on current H100, H200, and B200 systems, that all-reduce rides a fabric built for exactly this job, and TP scaling stays close to linear. Cross out of that domain onto InfiniBand between nodes, and each layer’s synchronization now waits on a slower, higher-latency hop. Cross onto plain PCIe with no NVLink at all, which describes most workstation cards including the RTX PRO 6000, and TP efficiency degrades enough that some deployments drop it entirely in favor of pipeline parallelism, trading per-request latency for throughput instead.
The node-boundary crossover
The number that matters for cost isn’t the raw GPU count your footprint math spits out. It’s whether that count sits at or under your node’s NVLink domain size. GLM-5.2’s roughly 510GB footprint clears at 3-4 cards of H200-class or B200-class hardware, which stays comfortably inside a single node on most current SKUs. That’s a clean deployment: one switch domain, near-linear scaling, no surprises.
Push a model past that ceiling and the math changes shape, not just size. DeepSeek V4 Pro runs 1600B total params against 49B active, and INT4 weights plus KV headroom put its footprint around 1.1TB. That clears 6-8 cards of H200-class hardware, right at the edge of a typical single-node domain, and clears it outright once you account for batching headroom. At that point you’re not choosing between GPU tiers anymore. You’re choosing between accepting an InfiniBand hop between chassis or hunting for fewer, bigger cards that keep the whole group on one node.
The messy failure
We’ve watched teams size a model like V4 Pro, land on an 8-GPU H200-class instance, and assume “8-GPU box” means “one coherent NVLink domain.” It doesn’t always. Some cloud SKUs advertise 8 GPUs per instance but only wire full NVSwitch connectivity across 4 of them, with the remaining 4 reachable over a slower on-board PCIe switch. Nothing on the spec sheet flags this. The failure surfaces during a load test: p99 latency triples because half the ring all-reduce traffic is silently running over the slower path, and the on-call engineer spends a night ruling out the model, the batch size, and the scheduler before finding the topology diagram.
The marketplace discount trap
Marketplace pricing compounds this. H200 NVL listings on the community tier run at a steep discount to secure-tier pricing, often 70% or more as of this writing, and glancing at price-per-card alone makes that discount look like pure savings for any TP job. Secure-tier contracts are usually the ones that guarantee your requested cards land inside one NVSwitch domain. Marketplace listings frequently make no such guarantee, because the underlying hardware wasn’t reserved or wired as one coherent pod. Take the discount for a 2-card job that fits inside one machine anyway and you likely keep the fast fabric and pocket the savings. Take it for a 6- or 8-card TP group and you may be buying cards that never shared a fast interconnect to begin with, and the discount evaporates the first time you measure tokens per second instead of dollars per card.
The decision rule
Run your model’s footprint through the calculator first, then check that number against your target node’s NVLink domain size before you check anything else. If the card count lands at or under a single node’s full domain (commonly 8 on current H100, H200, and B200 hosts), buy that configuration and expect scaling close to linear. If it clears that ceiling, don’t reach for more cards of the same class by default. Price out fewer, higher-capacity cards first, stepping up the H200 to B200 capacity class, to see whether you can pull the whole deployment back inside one node before you accept the InfiniBand tax.
What this can’t tell you is whether your specific provider’s advertised 8-GPU instance is genuinely one NVSwitch domain or two smaller ones stitched together under a single SKU name. That’s a question for the provider’s topology diagram and a real all-reduce benchmark on your own account, not for any pricing tracker.