NVIDIA's New Custom Networking Module

NVIDIA has a new custom form factor multi-NIC module for AI systems

NVIDIA has a new custom networking module set to power some of its next-generation GB200 servers. Years ago, we discussed the NVIDIA “Cedar” module on STH. This was a module that incorporated four NVIDIA ConnectX-7 NICs onto a single module for the NVIDIA DGX series. In the article’s title, we put “Cedar Fever”, and apparently that became a name for many in the industry.

With the NVIDIA GB200 NVL2, NVIDIA has had some challenges. One of them, most notably, is that it is asking its partners to increase the height of its NVIDIA MGX-based NVL2 platforms from 2U to 4U for thermal reasons. At OCP Summit 2024, it was certainly strange to see so many 2U designs, with all of the OEMs saying that NVIDIA asked them to change them to 4U designs.

Still, we found something sitting in many of the systems, and after taking a look, we realized that it is a new NVIDIA networking module. Nobody we talked to seems to know what this is called, so we are not going to have another “Cedar Fever” moment but we found it.

NVIDIA GB200 NVL2 Networking Module

When looking at a number of the NVIDIA GB200 NVL2 systems on the OCP Summit 2024 show floor, we saw the two grace CPUs, along with the two Blackwell GPUs. There were vertical modules for the BMC and M.2 SSDs as we have come to expect on NVIDIA MGX systems. Then, there was one more module.