NVIDIA RTX PRO 6000 Review: 600W Card Serves gpt-oss-120b at 3,064 tok/s

One NVIDIA RTX PRO 6000 Blackwell Server Edition, running at 600W, served gpt-oss-120b at 3,064 output tokens per second across 128 streams in StorageReview's vLLM test published Nov. 3, 2026. That is about a third of the 9,280 an ASUS ExpertCenter Pro ET900N G3 with a GB300 chip reached on the same harness.

The card's case does not rest on speed. It rests on 96 GB of memory in one PCIe slot. That capacity lets a single card hold OpenAI's 120-billion-parameter open model with room left for a large context cache. The RTX 5090 shares its GB202 chip but has 32 GB. This review sets NVIDIA's specifications against what StorageReview and Gamers Nexus measured on three editions of the card: the 600W Workstation Edition, the 300W Max-Q Workstation Edition and the passive Server Edition. Voltgrid did not test the card. Every figure below names its edition, its power limit and the software that produced it, because the outlets used different tools.

What's new

NVIDIA announced the RTX PRO 6000 Blackwell at GTC on March 18, 2025, as the successor to the RTX 6000 Ada. Partners began shipping in April and May 2025. NVIDIA's product page lists the Workstation Edition with 96 GB of GDDR7 memory with error correction, 1,792 GB/s of bandwidth, a 600 W maximum power draw and a dual-slot board 12.0 inches long on PCIe 5.0 x16.

Memory is the main change. The RTX 5090, built on the same GB202 chip, stops at 32 GB. Gamers Nexus counted 32 memory modules of 3 GB each on its teardown board, which makes 96 GB. The 512-bit bus is NVIDIA's figure.

The chip is a fuller GB202. NVIDIA enables 24,064 CUDA cores, against 21,760 on the RTX 5090, Gamers Nexus noted. NVIDIA quotes 4,000 AI TOPS, a theoretical FP4 figure that assumes sparsity. StorageReview's matrix-multiply test, which measures sustained rather than peak math, found 419 TFLOPS in BF16, 820 in FP8 and 1,634 in block-scaled FP4 on the 600W Server Edition.

The family is new too. The same silicon and memory ship as a 600W Workstation Edition with fans, a 300W Max-Q Workstation Edition for multi-card towers, and a passive Server Edition that StorageReview lists as configurable from 300 to 600W. NVIDIA also offers Multi-Instance GPU, which splits one card into up to four 24 GB slices, according to NVIDIA's datasheet.

What it solves

Memory bandwidth decides how fast a model writes once it fits. Memory capacity decides whether it fits at all. The RTX PRO 6000 answers the second question for 120B-class mixture-of-experts models without a second card.

StorageReview's numbers show what that means in practice. In its llama.cpp serving test inside a Dell Pro Precision 9 T2 tower, gpt-oss-120b with 8,192-token prompts left room for 64 parallel request slots and 22.5 GiB of context cache on one card. A single user got 182.7 tokens per second on the 600W Workstation Edition and 160.6 on the 300W Max-Q. Those are llama.cpp figures. They are not comparable with the vLLM throughput numbers elsewhere in this review.

The second problem it solves is the 32 GB wall. Gamers Nexus ran LM Studio in a test it labelled experimental, from a June 2025 video that predates the vLLM and llama.cpp runs above by more than a year. Its Workstation Edition and an RTX 5090 Founders Edition tied at about 81 tokens per second on an 8B model. On the 26 GB Mistral Small build, the 5090 fell to 17 against the PRO card's 42.4. On Gemma 3 27B the gap was 29 to 5, and on Llama 3.3 70B at Q4_K_S Gamers Nexus' figure put the PRO card 928% ahead. The pattern is the point: once the model leaves VRAM, the consumer card collapses.

The limit is equally clear. Above 96 GB the card needs company. StorageReview needed two Server Edition cards for MiniMax-M2.7 in NVFP4 and four for MiniMax-M3 and GLM-5.2. On GLM-5.2, four cards holding 384 GB managed 9 tokens per second for one user, against 35 on a single DGX Station.

Pros and Cons

The strongest evidence comes from one lab running one harness. StorageReview's Nov. 3 first look at AMD's Instinct MI350P put the RTX PRO 6000 Server Edition, at its full 600W in a Dell PowerEdge XE7740, beside the MI350P at 450W, an H200 NVL at 450W and an ASUS ExpertCenter Pro ET900N G3 DGX Station. NVIDIA's cards ran NVIDIA's vLLM 26.08 container on CUDA 13.4.1. On gpt-oss-120b, which ships in MXFP4 so every card ran the same weights, the RTX card reached 3,064 output tokens per second on 512-token chat prompts and 1,169 on 8,192-token prompts at 128 streams. The H200 NVL logged 3,376 and 1,825, the MI350P 5,057 and 2,101, and the DGX Station 9,280 and 5,023.

That test also shows the weakness. StorageReview measured the card's GDDR7 at 1,540 GB/s on reads and 1,483 GB/s on writes, against 3,291 GB/s for the MI350P and 6,869 GB/s for the DGX Station. On Mistral Small 3.1 24B at a single stream, the RTX card wrote 56 tokens per second in FP8 against 100 on the MI350P. StorageReview credits the AMD card's HBM3E memory for its single-stream lead. The RTX card kept its lead only where NVIDIA's NVFP4 builds ran at high concurrency, as on Llama 3.1 8B at 128 streams, 8,869 against 8,383, with NVFP4 on the RTX card and MXFP4 on the MI350P, two different FP4 formats.

The power limit matters less than its label suggests. StorageReview ran both Workstation Editions in the same Dell tower in its Oct. 20 review. MLPerf Client's Llama 3.1 8B decode on llama.cpp rose from 230.9 tokens per second on the 300W Max-Q to 252.8 on the 600W card, about 9% (Voltgrid's arithmetic from StorageReview's figures: 8.7% below the 600W card, 9.5% above the Max-Q). Time to first token fell from 0.082 to 0.059 seconds, 28% less. Across its suite, StorageReview found the 600W card added 7% to 20% on decode and 20% to 39% on prompt processing and image generation.

The heat is real. Gamers Nexus measured the 600W Workstation Edition's core at about 82C under a fixed render load, not the Server Edition in a chassis,, about 10C warmer than the RTX 5090 FE, with VRAM near 88C. Noise was about 1 dBA higher than the 5090 once Gamers Nexus adjusted for the noise floor. Its teardown found thermal paste instead of liquid metal, and some power inductors only partly covered by thermal pads. In StorageReview's Xeon tower, fitting the 600W card cost the processor 4% to 5% on all-core work and meant removing the 25GbE network card from the adjacent slot.

Who it's for

The first buyer is a developer or small team that wants a 120B-class mixture-of-experts model on one card in an existing tower. StorageReview's figures of 160.6 to 182.7 tokens per second for one user on gpt-oss-120b, depending on the edition, are interactive speeds. Sixty-four slots fit in memory for a team sharing the card.

The second is a studio that needs graphics, video and AI in one machine. StorageReview noted that the card's graphics, ray-tracing and video hardware give it a broader role than AMD's MI350P accelerator, and Gamers Nexus measured it 5% to 14% ahead of an RTX 5090 in games.

The third is a server buyer filling PCIe slots who plans to scale by card count. StorageReview's tests show the Server Edition handles 24B to 120B models on one card and large MoE models such as MiniMax-M2.7 on two cards and MiniMax-M3 on four.

It is not for a buyer whose target model is larger than 96 GB and who wants one device. Four cards ran GLM-5.2 at 9 tokens per second for one user in StorageReview's test, against 35 on a single DGX Station.

Why Memory Bandwidth Sets Decode Speed

A language model writes one token at a time, and each token needs the active weights read from memory again. Decode speed therefore tracks how fast memory can feed the chip more than how much math the chip can do, while memory capacity decides whether the model fits at all.

StorageReview's vLLM runs show the split. Its RTX PRO 6000 Server Edition measured 1,540 GB/s on reads and the GB300 DGX Station 6,869 GB/s, and the Station served gpt-oss-120b at 9,280 tokens per second to the card's 3,064. StorageReview credits the MI350P's HBM3E memory for its single-stream lead. The Spark's 273 GB/s and the Mac Studio's 1.2 TB/s are NVIDIA's and Apple's ratings, not measurements.

How it compares

Against the NVIDIA DGX Station, the numbers come from the same StorageReview harness. The ASUS-built GB300 Station's single GPU served gpt-oss-120b at 9,280 tokens per second to the RTX card's 3,064, and gpt-oss-20b at 22,041 to 7,937. It also has 748 GB of coherent memory, 252 GB of it HBM3e at 7.1 TB/s. The Station is rated at 1,600 W of system power by NVIDIA, and Dell lists its default Dell Pro Precision GB300 tower at $160,195 on Dell.com as of Nov. 1. Those are three different units: the 9,280 tok/s came from ASUS's machine, the wattage is NVIDIA's rating and the price is Dell's. Our DGX Station review cited an earlier StorageReview RTX PRO 6000 result from a different setup. The figures here are the lab's newest, from the Server Edition in a server.

Against the NVIDIA DGX Spark, the trade is capacity for bandwidth. The Spark holds 128 GB of unified memory but moves it at 273 GB/s, against the Workstation Edition's rated 1,792 GB/s. Its power supply is rated 240 W, and the 128 GB, 4 TB model was listed at $6,950 in a copy of NVIDIA's marketplace page cached Oct. 31, with stock unverified. StorageReview's new runs did not include a GB10 machine, so this review sets no tokens-per-second ratio between the two. Our GB10 comparison has the Spark-class vLLM figures.

Against the Apple Mac Studio with M5 Ultra, the Mac offers up to 512 GB of unified memory at 1.2 TB/s in a box with a 480 W maximum continuous power rating. MacRumors, reading Apple's configurator, put the 512 GB configuration at a minimum of $22,299 on Oct. 28. The Mac holds models the RTX card cannot, but its published LLM tests use MLX and LM Studio on different models from the ones StorageReview ran on the RTX card, so no like-for-like speed comparison exists yet.

Price is the open variable. NVIDIA's US marketplace listed the Workstation Edition at $16,000, out of stock, when last read on Aug. 10, 2026. That listing has not been rechecked since, so treat the figure as dated. Gamers Nexus paid $8,000 for its card in 2025. In StorageReview's Dell.com configuration, which used the 300W Max-Q card, the card made up $19,090 of a roughly $28,300 tower, and Dell charged the same for the 300W and 600W editions.

What to watch

StorageReview said it will rerun the MI350P at its full 600W in a Dell PowerEdge XE7745 and repeat its two-card tests on ROCm 10.1. It gave no date. Those runs will show whether the RTX card's gap to AMD's PCIe card narrows or widens. StorageReview's Oct. 20 Dell Pro Precision 9 T2 review also plans a pass with external power and temperature logging to explain the CPU slowdown it saw with the 600W card in the Xeon tower.

The rivals have dates of their own. DGX Station for Windows is due from partners in December 2026, Apple's first 512 GB M5 Ultra orders are due to arrive on Dec. 3, and NVIDIA's quarterly results call is set for Nov. 17.

The last open item is a current US price for the RTX PRO 6000 Workstation Edition. Until NVIDIA or a major retailer posts one, any cost-per-token comparison with the DGX Spark or Mac Studio rests on an August reading.