← All questions
Inference & ServingMedium· Hardware Architecture· Interconnects

Walk me through the bandwidth hierarchy inside a DGX H1...

The bandwidth hierarchy inside a DGX H100 node is structured into three primary tiers. First, PCIe Gen 5 handles CPU-to-GPU traffic at 128 GB/s bidirectional. Second, NVLink 4.0 provides 900 GB/s per GPU for direct GPU-to-GPU communication. Finally, NVSwitch acts as the critical component that provides a non-blocking all-to-all crossbar. Without NVSwitch, GPU pairs lacking a direct NVLink lane would contend for bandwidth, significantly reducing performance.

Code playground (Python)