Summary

NVIDIA says its Vera Rubin NVL72 achieved up to 3.7 times the throughput of GB300 NVL72 in preview submissions to MLPerf Inference v6.1. The results combine new hardware with software optimisations for large-scale AI inference.

NVIDIA says its first preview submission using the Vera Rubin NVL72 delivered up to 3.7 times the throughput of its GB300 NVL72 system in MLPerf Inference v6.1 results released on September 16, 2026. The comparison covered Qwen3-VL and DeepSeek-R1, two demanding language and vision-language workloads in the benchmark suite.

The figures come from NVIDIA platform entries in the MLPerf Inference v6.1 closed division. They describe a combined hardware-and-software configuration rather than a hardware-only comparison.

Contents

What the MLPerf preview showed

On Qwen3-VL, NVIDIA reports up to 3.7 times higher throughput for Vera Rubin NVL72 than GB300 NVL72 across offline, server and interactive scenarios. The Vera Rubin submission used vLLM together with NVIDIA’s Dynamo open-source inference framework.

On DeepSeek-R1, the Vera Rubin system delivered up to 2.5 times the throughput of GB300 NVL72, using NVIDIA’s TensorRT-LLM software library. Throughput measures how much inference work a system completes over time; for generative AI systems, higher throughput can mean more output tokens or user requests handled by the same infrastructure.

NVIDIA attributes the results to full-stack codesign. That means the performance comes from the interaction between the processors, memory, interconnects and inference software used in the test configuration.

How Vera Rubin improves inference throughput

Vera Rubin’s enhanced Tensor Cores and Transformer Engine are designed to accelerate both major stages of model inference. During prefill, the system processes the input context. During decode, it generates output tokens sequentially. Improving both stages is important for workloads ranging from ordinary text generation to longer reasoning tasks.

NVIDIA also used NVFP4 precision to reduce the memory footprint of model weights, attention data and the key-value cache. The key-value cache stores information from earlier tokens so that a model can generate later tokens without recomputing the entire context. Using less memory can allow more data to remain close to the compute units and increase the number of requests handled by a system.

The Vera Rubin submissions also relied heavily on disaggregated serving, which separates prefill and decode workloads, and on large-scale expert parallelism for mixture-of-experts models such as DeepSeek-R1 and Qwen3-VL. These techniques distribute different parts of inference across the system so that each stage and model component can be operated more efficiently.

The NVL72 scale-up domain provides the rack-level connection between the GPUs. NVIDIA says its sixth-generation NVLink and NVLink Switch deliver 10 times higher packet rates and three times lower latency than off-the-shelf Ethernet. A fast, low-latency interconnect is important when many GPUs must exchange data during one inference task.

GB300 demonstrates rack-scale scaling

The same MLPerf release also highlighted the scaling efficiency of NVIDIA’s GB300 NVL72. A submission using four racks, or 288 GPUs, achieved 99% scaling efficiency on DeepSeek-R1 in the offline scenario. It scaled from a single 72-GPU rack, with throughput increasing nearly in proportion to the added hardware.

Scaling efficiency measures how much additional throughput is obtained when more GPUs are added. High efficiency helps keep infrastructure productive as systems expand, because the cost and power of additional hardware are more closely matched by useful inference capacity.

On the WAN 2.2 text-to-video benchmark, GB300 NVL72 reached 0.65 720p videos per second at 5.7 seconds per video. NVIDIA reported this as nine times higher throughput and 7.5 times lower latency than a single node.

NVIDIA also reported that software changes improved GB300 NVL72 performance on Qwen3-VL by up to 1.6 times compared with its MLPerf Inference v6.0 results. The company attributed those gains to lower key-value-cache precision, additional kernel fusion, improved kernels and disaggregated serving using vLLM and Dynamo.

The Vera Rubin figures are preview-submission results for the named MLPerf workloads, and NVIDIA presents the performance gains as a product of hardware and continuing software optimisation. The company says further post-submission results for GPT-OSS-120B and DLRMv3 had not yet been verified by MLCommons.

Sources