NVIDIA Vera Rubin MLPerf Debut Sets the Pace, but Preview Status Matters

NVIDIA entered MLPerf Inference v6.1 with Vera Rubin NVL72, reporting up to 3.7 times the throughput of its GB300 NVL72 predecessor. The NVIDIA Vera Rubin MLPerf debut gives infrastructure buyers their first peer-reviewed view of the company’s next rack-scale AI platform. It also places an important limitation beside the headline: Vera Rubin remains a preview system.

That distinction frames the real contest. NVIDIA is not merely comparing one GPU generation with another. It is arguing that tightly integrated racks can turn new silicon, networking, and serving software into better inference economics.

AMD’s broadest MLPerf submission to date shows why that argument now faces pressure. AMD posted competitive results on available hardware, expanded its workload coverage, and highlighted deployments reaching 512 accelerators. NVIDIA leads the selected Vera Rubin comparisons, but customers must decide whether a preview lead translates into deployable value.

NVIDIA Vera Rubin MLPerf Results Establish an Early Lead

The debut establishes a large generational performance gain on two demanding models, although it does not yet represent a shipping production platform.

The results arrived on September 16, 2026, as part of the MLPerf v6.1 release. MLPerf Inference measures how systems serve trained AI models under standardized accuracy and latency conditions.

MLCommons reported a record 30 participating organizations in this round. Five new processors or accelerators appeared, including AMD’s available MI350P and Intel’s Arc Pro B70. NVIDIA Rubin and Vera Rubin NVL72 entered in preview.

NVIDIA submitted Vera Rubin results for DeepSeek-R1 and Qwen3-VL-235B. DeepSeek-R1 tests reasoning-model serving, while Qwen3-VL combines language and visual inputs for tasks such as product classification.

The company’s Vera Rubin results show the largest relative gain in Qwen3-VL’s interactive scenario. There, one Vera Rubin NVL72 delivered 1,307 queries per second. A comparable GB300 NVL72 result reached 349 queries per second.

That produces the headline increase of roughly 3.7 times. The interactive scenario also enforces a 1.5-second latency limit, making the result more relevant than unconstrained batch throughput for responsive applications.

The advantage was smaller, but still substantial, in other Qwen3-VL scenarios. Vera Rubin reached 2,393 samples per second offline, compared with 1,305 for GB300. Its server result reached 2,323 queries per second, compared with 1,210.

Those results equal about 1.8 times higher offline throughput and 1.9 times higher server throughput. The difference shows why a single maximum multiplier cannot describe the entire system.

On DeepSeek-R1, Vera Rubin recorded 652,750 tokens per second in the interactive scenario. A 72-GPU GB300 NVL72 result reached 260,098 tokens per second under the same published latency constraints.

That comparison supports NVIDIA’s claim of up to 2.5 times higher DeepSeek-R1 throughput. NVIDIA used TensorRT-LLM for this workload, while its Qwen3-VL submission used vLLM with NVIDIA Dynamo.

The published performance table also reports accuracy thresholds beside each result. The Vera Rubin DeepSeek-R1 entry targets 99 percent of FP16 accuracy, with an exact-match score of 81.9132 percent.

For Qwen3-VL, the target was 99 percent of BF16 quality. The benchmark required a category hierarchical F1 score of at least 0.7824 using a Shopify product catalog dataset.

These gates matter because faster inference has limited value if aggressive numerical compression damages model output. MLPerf requires submissions to meet defined quality thresholds before their performance numbers count.

Still, the debut covers only two benchmark models. It does not establish leadership across every language, recommendation, video, speech, or retrieval workload in MLPerf Inference v6.1.

That narrower scope is not unusual for preview hardware. It does mean buyers should read the result as an early system signal, not a universal verdict.

Why More Throughput Changes AI Inference Economics

Inference performance matters financially when additional throughput serves more useful requests without proportionally increasing hardware, power, or operational complexity.

Training produces a model, but inference runs whenever that model answers a user. A consumer assistant, coding agent, search service, or document system can trigger many inference calls during one task.

Reasoning agents intensify that pattern. They generate intermediate tokens, call tools, examine results, and revise their plans. One visible answer can require several model passes behind the interface.

That behavior makes tokens per second economically important. A faster system can serve more concurrent requests, reduce queues, or deliver longer reasoning traces within the same time window.

However, maximum throughput is only one part of the calculation. Buyers also need acceptable response latency, consistent accuracy, high utilization, manageable power demand, and reliable software.

MLPerf separates several deployment scenarios for this reason. The offline scenario emphasizes bulk processing. The server scenario models variable incoming requests, while the interactive scenario imposes tighter responsiveness requirements.

Vera Rubin’s strongest relative showing appears in that latency-sensitive interactive setting. This suggests its architectural improvements become especially valuable when a system cannot hide delays behind large batches.

The outcome matters for products that charge for access, consumption, or completed work. If one rack handles more requests at a required service level, the operator gains more capacity from that infrastructure.

Yet throughput does not automatically equal revenue. Demand must exist, software must keep accelerators occupied, and the complete service must avoid bottlenecks outside the model server.

Storage, network fabrics, databases, safety filters, and application logic can all limit delivered performance. A benchmark isolates the system under test more cleanly than most production environments can.

Cost per token is also more complex than dividing equipment cost by peak throughput. It includes electricity, cooling, networking, maintenance, financing, software engineering, downtime, and depreciation.

NVIDIA says Vera Rubin lowers token costs, but the v6.1 release does not publish a complete production total-cost comparison. The benchmark provides performance evidence, not a full purchasing model.

Infrastructure teams should therefore translate these results into their own traffic profiles. A visual product-search service may care about Qwen3-VL server throughput. A reasoning platform may prioritize DeepSeek-R1 latency and token generation.

The result becomes more valuable when the benchmark resembles the buyer’s workload. It becomes less decisive when model architecture, request length, batching behavior, or service constraints differ substantially.

This is the economic pressure NVIDIA’s competitors face. They do not need to win every chart, but they must offer enough deployable performance, software maturity, and flexibility to produce a better overall business case.

Rack-Scale Codesign Is the Performance Mechanism

Vera Rubin’s advantage comes from treating 72 GPUs, CPUs, memory, networking, and inference software as one coordinated system.

An NVL72 rack connects 72 accelerators inside a high-bandwidth scale-up domain. Scale-up networking lets those accelerators cooperate as parts of one large system instead of behaving like isolated servers.

NVIDIA says sixth-generation NVLink and NVLink Switch provide 10 times higher packet rates than standard Ethernet. The company also claims three times lower latency within the relevant comparison.

Those interconnect claims come from NVIDIA and should not be treated as independent network benchmarks. They nevertheless explain the design choice behind the submitted system.

Large reasoning and vision-language models move substantial amounts of data during inference. Model weights, activations, attention states, and routing decisions must reach the right computing resources quickly.

Vera Rubin uses HBM4, a high-bandwidth memory generation placed close to the GPU. NVIDIA’s rack configuration includes 72 Rubin GPUs and 36 Vera CPUs, according to its published architecture.

The software then divides work across those resources. Disaggregated serving separates prefill from decode, allowing different resources to specialize in distinct phases of text generation.

Prefill processes the user’s input and builds the model’s internal attention state. Decode generates the response token by token, often under different memory and latency constraints.

Separating those phases can improve utilization when software assigns each one to suitable resources. It can also create coordination overhead, so fast interconnects and effective scheduling become critical.

NVIDIA’s submissions used large-scale expert parallelism for mixture-of-experts models. This model design activates only selected expert layers for each token instead of running the entire network every time.

The approach can reduce computation, but it creates irregular communication. Tokens must be routed to the correct experts, processed, and returned without letting network delays consume the savings.

Lower-precision arithmetic forms another part of the mechanism. NVIDIA used NVFP4, a four-bit numerical format intended to reduce memory use and increase computation throughput.

The system applied reduced precision to weights, attention operations, and the key-value cache. A key-value cache stores prior attention information so the model does not recompute the complete conversation for every new token.

Lower precision allows more data to fit in memory and reduces movement between computing units. The accuracy thresholds in MLPerf provide a guardrail against unacceptable quality loss.

The final performance figure is therefore not a pure measurement of Rubin silicon. It reflects processors, memory, networking, numerical formats, kernels, serving frameworks, and benchmark-specific tuning.

That is central to NVIDIA’s strategy. The company wants customers to evaluate the entire platform because its competitive advantage extends beyond individual accelerator specifications.

The strategy can create benefits for operators seeking one supported stack. It can also increase dependence on NVIDIA’s tightly connected hardware and software environment.

For buyers, that dependency is not automatically negative. A coordinated platform can reduce integration work and produce predictable performance across qualified systems.

The tradeoff appears when customers want greater hardware choice, portable software, or independent control over individual components. That is where AMD’s competing approach becomes relevant.

AMD Pressures the Platform Story From Available Hardware

AMD’s v6.1 showing does not erase Vera Rubin’s selected leads, but it prevents NVIDIA from owning the broader inference narrative uncontested.

AMD expanded from three model families in MLPerf Inference v6.0 to six in v6.1. Its results covered language, reasoning, recommendation, and text-to-video workloads on MI355X, MI350X, and MI350P accelerators.

The company’s MLPerf submission highlights competitive eight-GPU performance on GPT-OSS-120B. AMD says MI355X led selected NVIDIA B200 and B300 entries in offline and server scenarios.

AMD also reported gains from software work on unchanged MI355X hardware. Its eight-GPU GPT-OSS-120B server throughput improved by 38 percent compared with the prior round.

Offline throughput rose by 28 percent, while Wan 2.2 single-stream performance improved by 70 percent. These are AMD’s comparisons of its submitted v6.1 and v6.0 results.

The same pattern supports one of NVIDIA’s core arguments. Hardware does not determine the lifetime value of an inference system by itself. Software improvements can unlock more useful work after deployment.

AMD reached 95 percent scaling efficiency with 72 MI355X GPUs on GPT-OSS-120B. Scaling efficiency measures how closely throughput growth follows the addition of accelerators.

Crusoe then submitted a 512-GPU AMD system. It produced 5.75 million offline tokens per second on GPT-OSS-120B and 2.90 million on DeepSeek-R1, according to AMD.

That was the largest accelerator count submitted to MLPerf Inference at the time. It also illustrates a competing route to scale based on large clusters and an open software environment.

The comparison with Vera Rubin is not direct. Different submissions can use different models, system sizes, scenarios, and approved optimizations.

A 512-accelerator aggregate result does not establish better per-rack economics than a 72-GPU Vera Rubin system. Likewise, Vera Rubin’s Qwen3-VL lead does not settle AMD’s competitiveness on GPT-OSS-120B.

This is a common challenge when reading MLPerf. Every vendor can emphasize the workloads, scenarios, and system sizes that present its strongest case.

MLCommons reduces that freedom through standard rules, accuracy gates, result review, and published configuration details. It cannot make unlike systems identical.

The most useful comparisons hold the model, scenario, latency target, accuracy requirement, accelerator count, and division constant. Differences outside those variables should remain visible.

Competition also extends beyond NVIDIA and AMD. Google participated in v6.1, Intel introduced Arc Pro B70 results, and multiple cloud providers submitted partner systems.

Lambda’s inference results demonstrate how system builders can generate further gains using the same accelerator generation. Its four-GPU Blackwell Ultra system improved nearly 9 percent over v6.0.

Lambda also used an open-division submission to run an edge-agentic workload with Kimi K2.6 on datacenter hardware. The model contains more than one trillion parameters, according to Lambda.

These entries reinforce a broader point. The competitive unit is shifting from a processor toward a configured service stack that includes models, frameworks, orchestration, and infrastructure.

NVIDIA currently offers the most integrated version of that stack. AMD is trying to weaken the assumption that integration requires an NVIDIA-only platform.

The pressure on NVIDIA is therefore not simply to remain fastest. It must show that its performance lead produces enough operational value to justify the commitment to its architecture.

What the Preview Results Do Not Establish

The strongest claim is verified benchmark throughput, while claims about cost, revenue, availability, and broad workload leadership require additional evidence.

MLCommons identifies Vera Rubin and Vera Rubin NVL72 as preview entries. That label tells readers the platform is not being represented like the available AMD MI350P in this results round.

Preview status does not make a result invalid. MLPerf still publishes the configuration and applies its benchmark process. It does limit what buyers can infer about commercially deployed systems.

Production hardware can face constraints that do not appear in an engineering submission. Firmware maturity, failure rates, component supply, cooling, installation time, and fleet management all affect real capacity.

The Vera Rubin submission also came from NVIDIA and partner Nebius. Broader reproduction across independent system builders would provide stronger evidence that the performance transfers across deployments.

NVIDIA’s earlier-generation ecosystem offers a favorable precedent. Nineteen partners participated with NVIDIA platforms in v6.1, including eight using multi-node Blackwell NVL72 systems.

That breadth suggests NVIDIA knows how to move optimized designs through an extensive supplier network. It does not guarantee that Vera Rubin’s preview configuration will arrive unchanged or on schedule everywhere.

Workload coverage creates another uncertainty. The platform debuted on DeepSeek-R1 and Qwen3-VL, two workloads suited to its rack-scale design and software stack.

It did not publish Vera Rubin results across GPT-OSS-120B, recommendation, speech, image generation, RAG, or Wan 2.2 in the closed division. Future rounds should show whether the lead generalizes.

NVIDIA also cited post-deadline performance improvements on GPT-OSS-120B and DLRMv3. The company explicitly said MLCommons had not verified those later results.

Those numbers should remain outside direct benchmark conclusions until they pass the same review process. Vendor testing can indicate direction, but it does not carry identical evidentiary weight.

Power data is another missing piece in the headline comparison. Throughput per rack matters, but operators often face fixed electrical capacity before they exhaust floor space.

A faster rack can still complicate deployments if it requires denser power delivery or more demanding liquid cooling. Performance per watt will determine how much useful work fits within a facility’s limit.

NVIDIA argues that more throughput lowers cost per token. That conclusion is plausible when other factors remain stable, but the published result does not provide a complete cost accounting.

Purchase terms, utilization, power, maintenance, financing, and software labor can move the final number. Buyers need measured production data from their own workloads.

The benchmark also cannot guarantee end-user responsiveness. Model serving may run quickly while retrieval, tool calls, databases, or external APIs create most of the delay.

MLPerf v6.1 begins addressing this gap with new retrieval-augmented generation and edge-agentic tests. More than half of submitters used its API-centric harness, which applies a client-server architecture.

MLCommons says that harness will support a transition toward MLPerf Endpoints for datacenter testing. The change should bring measurements closer to deployable services rather than isolated model engines.

That evolution could strengthen or weaken NVIDIA’s platform case. A tightly integrated stack should benefit from full-service measurement, but external dependencies can dilute raw accelerator advantages.

The prudent reading is specific. Vera Rubin posted a notable, reviewed preview debut on DeepSeek-R1 and Qwen3-VL. It has not yet established universal leadership or complete production economics.

Three Signals Will Determine Whether the Lead Holds

Vera Rubin’s early advantage becomes strategically important only if shipping systems reproduce it across more workloads and under measurable operating constraints.

The first signal is broad production availability. Customers should watch for cloud instances, partner systems, and installed racks that expose configuration details comparable with the preview submission.

A successful rollout would show that NVIDIA can convert an optimized engineering result into repeatable infrastructure. Delays or materially different configurations would weaken that conclusion.

Availability should include more than shipment announcements. Buyers need stable software releases, qualified networking, service procedures, and evidence that systems sustain performance over long operating periods.

The second signal is the next independent benchmark round. Vera Rubin needs results beyond DeepSeek-R1 and Qwen3-VL, especially on retrieval, recommendation, video, and additional language models.

Broader coverage would support NVIDIA’s claim that one platform can handle diverse inference workloads. Narrow coverage would leave more room for specialized accelerators and competing clusters.

Future MLPerf Endpoints results will be particularly revealing. They should measure API-facing systems that more closely resemble services purchased and operated by enterprises.

Watch whether Vera Rubin’s raw throughput survives the addition of request orchestration, retrieval components, and realistic service boundaries. A smaller lead would suggest bottlenecks moved elsewhere.

The third signal is competitive software progress. AMD’s v6.1 improvements show that installed accelerators can gain substantial performance without a new hardware generation.

If ROCm, open serving frameworks, and partner systems continue improving quickly, buyers may accept lower peak performance in exchange for flexibility or existing infrastructure compatibility.

If NVIDIA’s software improves at the same pace, Vera Rubin’s initial lead may compound. The company already reported a 1.6-times Qwen3-VL gain on GB300 between v6.0 and v6.1.

Software velocity therefore matters twice. It raises current performance and changes how customers estimate the useful life of an expensive system.

Procurement teams should request results for their exact models, context lengths, batch patterns, and latency targets. They should also test failures, upgrades, and workload changes.

Developers should watch which optimizations reach common frameworks without extensive custom work. A benchmark-tuned technique creates limited value if ordinary teams cannot deploy or maintain it.

AI product leaders should connect infrastructure metrics to user outcomes. Faster inference can support lower latency, more reasoning steps, higher concurrency, or longer multimodal requests.

The NVIDIA Vera Rubin MLPerf result makes a credible opening case for rack-scale codesign. Its 3.7-times peak comparison is meaningful, but it belongs to a defined scenario.

The next decision is not whether the preview benchmark looks fast. It clearly does. The decision is whether shipping systems preserve that advantage after power, software, availability, and real workloads enter the equation.

Ask vendors to reproduce your most demanding service-level target before choosing a platform. Then compare sustained throughput, latency, accuracy, utilization, and operational effort together. That evaluation will reveal whether Vera Rubin’s benchmark lead becomes durable inference economics or remains an impressive preview result.