GB200 NVL72 / GB200 NVL72 specs
GB200 NVL72 specs
What you get in one rack, what each number means for your workload, and where each figure comes from.
The spec table
| Item | Per rack | Source |
|---|---|---|
| GPUs | 72 Blackwell | nvidia.com |
| CPUs | 36 Grace (2,592 Arm Neoverse V2 cores) | nvidia.com |
| GPU memory (HBM3e) | Up to 13.4 TB total | nvidia.com |
| HBM bandwidth | Up to 576 TB/s | nvidia.com |
| CPU memory | Up to 17 TB LPDDR5X | nvidia.com |
| NVLink bandwidth in the rack | 130 TB/s | nvidia.com |
| FP4 Tensor Core | 1,440 PFLOPS sparse, 720 dense | nvidia.com |
| FP8 Tensor Core | 720 PFLOPS sparse | nvidia.com |
| FP16 / BF16 Tensor Core | 360 PFLOPS sparse | nvidia.com |
| Network per GPU (cloud builds) | 400 Gb/s InfiniBand | learn.microsoft.com |
| Layout | 18 compute trays (4 GPUs + 2 CPUs each), 9 NVLink switch trays | learn.microsoft.com |
| Cooling | Liquid-cooled | nvidia.com |
Figures are for a full rack, from NVIDIA's GB200 NVL72 page unless marked. The per-GPU network speed and tray layout come from Microsoft's documentation for the ND GB200 v6 series, which runs on GB200 NVL72. Sparse figures assume structured sparsity; dense is half.
How the rack is laid out
A GB200 NVL72 is 18 compute trays and 9 NVLink switch trays. Each compute tray holds two GB200 superchips, so four Blackwell GPUs and two Grace CPUs. The switch trays sit in the middle of the rack and give every GPU a direct NVLink path to the other 71 (Microsoft).
Clouds often split the rack up. Azure exposes it as 18 VMs of four GPUs; AWS sells 36- and 72-GPU UltraServers (AWS). If your job needs all 72 GPUs in one domain, ask for whole racks and confirm the scheduler hands you the domain intact.
Memory
Up to 13.4 TB of HBM3e across 72 GPUs at 576 TB/s, plus up to 17 TB of LPDDR5X on the Grace CPUs that the GPUs can reach over NVLink-C2C (NVIDIA). Azure lists 192 GB of HBM per GPU for its ND GB200 v6 VMs (Microsoft). If your model or KV cache doesn't fit, look at GB300 NVL72, which carries 288 GB per GPU.
Networking
Inside the rack, NVLink carries 130 TB/s. Between racks, NVIDIA's reference design uses Quantum-X800 InfiniBand or Spectrum-X800 Ethernet with BlueField-3 DPUs (NVIDIA). The clouds we checked run 400 Gb/s per GPU: InfiniBand at CoreWeave and Azure, EFA at AWS and RoCE at Google. Ask which fabric your racks are on and whether it's non-blocking at your size.
Power and cooling
The rack is liquid-cooled and was designed around roughly 120 kW (eeNews Europe, Oct 2024), several times a normal colocation rack. That is the main reason most teams rent GB200 NVL72 instead of hosting their own. More on power and cooling.
Questions
How much memory does a GB200 NVL72 have?
Is GB200 the same as B200?
How many FLOPS is a GB200 NVL72?
RackShore runs this site and may be paid a referral fee by the provider you choose. Buyers pay us nothing. Spot a mistake? Tell us.