NVIDIA HGX Rubin NVL8 Memory Bandwidth Cut 26% to 130 TB/s

NVIDIA's HGX Rubin NVL8 eight-GPU board now lists 2 TB of HBM4 at 130 TB/s, down from 2.3 TB and 176 TB/s in the preliminary figures, ServeTheHome reported on Oct. 31.

That is a cut of about 26% in memory bandwidth for the 8-GPU system, ServeTheHome calculates, against a cut from 22 to 19.2 TB/s for a single Rubin GPU. NVIDIA's own HGX page, read on Nov. 1, carries the 2 TB and 130 TB/s figures. The company has not said why it changed them.

The NVL8 is NVIDIA's eight-GPU HGX board, a smaller building block than the 72-GPU rack. Memory is the part that shrank: NVIDIA's HGX page lists 2 TB of HBM4 for the board, against 2.3 TB before, and the company has not said how the memory is configured.

What Changed in the HGX Rubin NVL8 Specs

NVLink Switch bandwidth for the board fell from 28.8 TB/s to 24 TB/s, and bidirectional scale-out networking rose from 1.6 TB/s to 1.8 TB/s, according to ServeTheHome's comparison with the preliminary table. NVIDIA's page lists 400 PFLOPS of sparse NVFP4 inference, 270 PFLOPS of dense NVFP4 training and 130 PFLOPS of dense FP8/FP6 training. ServeTheHome puts the preliminary training figures at 280 PFLOPS and 140 PFLOPS. Inference is unchanged at 400 PFLOPS.

The page applies to both the HGX Rubin NVL8, which pairs the GPUs with an x86 CPU, and the HGX Vera Rubin NVL8, which uses NVIDIA's Vera CPU.

Earlier Rubin NVL72 Spec Change

Voltgrid reported on Oct. 26 that the Vera Rubin NVL72 page listed 19.2 TB/s of HBM4 per GPU instead of up to 22 TB/s, and 3 TB/s of NVLink per GPU instead of 3.6 TB/s. NVIDIA's NVL72 page still lists those values, along with 1,400 TB/s of memory bandwidth and 216 TB/s of NVLink for the rack. Memory capacity is unchanged at 288 GB per GPU and 20.7 TB per rack.

ServeTheHome adds two things to that account. It says NVIDIA posted the revised specifications in September, and that the earlier figures were labelled preliminary and "subject to change." It also converts the per-GPU number into a pin speed: 22 TB/s across eight stacks implies about 10.75 Gbps per pin, and 19.2 TB/s implies about 9.375 Gbps. For comparison, the outlet notes AMD runs HBM4 on its MI455X at 7.6 Gbps per pin.

How HBM4 Bandwidth Is Set

A GPU's memory bandwidth is the width of its memory connection times the speed of each wire. HBM4 widens each stack's interface to 2,048 bits, and JEDEC's standard sets up to 8 Gb/s per pin, so NVIDIA's pin speeds, which ServeTheHome calculates at 10.75 Gb/s for the earlier 22 TB/s target and 9.375 Gb/s for today's 19.2 TB/s, sit well above the standard.

On the eight-GPU HGX Rubin NVL8 board, NVIDIA now lists 2 TB of HBM4 at 130 TB/s, down from 2.3 TB and 176 TB/s on its preliminary page. That is less memory than the 2.1 TB NVIDIA lists for HGX B300, and NVIDIA now calls the bandwidth 2.1 times HGX B200's, where its preliminary page claimed 3 times. When decoding is limited by memory, less bandwidth lowers the ceiling on tokens per second.

Why the NVL8 Bandwidth Cut Matters for Inference Buyers

For buyers sizing inference clusters, memory bandwidth often sets the token rate, so the NVL8 loses more than the rack does. The inference compute rating did not move, so the same 400 PFLOPS now sits beside less memory throughput. ServeTheHome says the effect on real workloads is unclear, because not every workload is limited by memory or by NVLink. Planners who sized capacity around the January figures should check the revised ones with their server vendor before they order.

Because the figures were labelled preliminary, any vendor quote should state which specification date it uses.

What NVIDIA Has Not Confirmed

NVIDIA has not published a note explaining the reduction. ServeTheHome assumes memory partners could not deliver the original pin speeds or yields, but that is its assumption. The Jan. 5 NVIDIA technical blog gave up to 22 TB/s and 3.6 TB/s per GPU and did not list NVL8 system figures, while NVIDIA's earlier HGX page, footnoted as preliminary information subject to change, listed the 2.3 TB and 176 TB/s baseline. Google Cloud said on Oct. 21 that its Vera Rubin NVL72 instances reach general availability in November, and neither company has said which bandwidth figures those instances will run at.