Ahead of our NVIDIA DGX Station (GB300) series, where we have been testing three systems that should have STH reviews go live in October, we found something that others simply have not mentioned before. The networking and topology of these systems are completely different from what you might think. I would go so far as to say that it makes the NVIDIA solution very different from what AMD is proposing with the AMD Threadripper Halo Station, and what many system builders have been offering with NVIDIA RTX Pro 6000 Blackwell Edition GPU servers, even for scale-out. It turns out that, unlike the NVIDIA GB10 systems, we can get 800Gbps from the two 400G ports on the GB300 AI Station platform versus ~230Gbps concurrently on the GB10 system’s two 200G ports. We also found a data path that cannot get close to that, and there is a good reason as to why. NVIDIA actually pulled a “baller” move on the system’s architecture and networking that will put it in a different class of system for neoclouds and enterprises.
We have seen reviews pass off topology diagrams that look like this one from Gigabyte (but the MSI and other ones are similar):
Most people see that and think that the topology is fairly straightforward and works like a workstation. Instead, NVIDIA designed this much more like a scale-out GPU server than you might otherwise think. This is one of the most fascinating network and system architectures we have seen in something designed to sit desk-side.
After over $350K of gear, we have tested this thoroughly and have a good sense of what is going on, so let us get to it.
The Network Performance Test We Did Not Expect
Let us get to the results and see what we found when we swept networking performance using RDMA on the systems. First, starting with what we would consider a “baseline” of an AMD Ryzen Threadripper (Pro) style system, which is similar to what you would expect from an Intel Xeon workstation with a GPU:
Here, you can see the CPU with a PCIe link to the AMD Radeon AI Pro R9700 32GB card via PCIe. Empty PCIe Gen5 x16 links can be used for additional R9700 cards, other GPUs, or up to a 400Gbps NIC. That is all dictated by the AMD Ryzen Threadripper series processor.
This looks like what AMD showed off with its Threadripper Halo Station platform.
Now the NVIDIA DGX Station (GB300) is different. Here is the standard lstopo output that you can compare to the above Threadripper system since that is what is driving the big difference in performance: