NVIDIA Vera Rubin NVL72 Claims 67x Better Performance per Dollar, but the Operating Point Matters
NVIDIA Vera Rubin NVL72 has posted a claimed 67x performance-per-dollar advantage over GB300 NVL72 in an early agentic inference test. That number is real within the selected benchmark comparison, but it is not a universal multiplier. At more common serving speeds, the measured advantage was much smaller.
The September 14 results give infrastructure buyers their first independent look at Rubin under a long-context, multi-turn workload. They also make NVIDIA CEO Jensen Huang’s earlier performance forecast look conservative. Yet the largest multiplier depends on a demanding operating point where the compared Blackwell configuration approaches its performance limit.
The central contest is therefore not simply Rubin against Blackwell. It is the headline claim against the performance that operators can reproduce across models, serving engines, latency targets, and production traffic. NVIDIA appears to have moved that entire performance curve, but the distance varies sharply along it.
NVIDIA Vera Rubin NVL72 Moves From Forecasts to Measured Agent Work
The important change is that Rubin now has measured agentic inference results, not just architectural specifications or NVIDIA projections.
SemiAnalysis published the first reviewed Rubin results from AgentX, its benchmark for long-context coding-agent traffic. The tests used production-shaped sessions with accumulated context, tool pauses, prefix reuse, and bursts of parallel subagents.
That traffic differs from a conventional chatbot benchmark. A basic chat request often contains one prompt and one response. An agent can conduct hundreds of turns while calling tools, consulting subagents, and repeatedly sending an expanding conversation history.
These patterns place different demands on an inference system. Long histories consume key-value cache capacity, which stores previously computed attention data. Reused prefixes reward efficient caching, while subagent bursts test scheduling and concurrency.
The public AgentX methodology is based on 393 opt-in coding sessions containing 135,282 requests. Its median reconstructed request has 142,016 input tokens and 444 output tokens.
AgentX removes prompts, code, tool arguments, and tool results. It preserves request lengths, timing, shared-prefix relationships, and subagent structure, then substitutes deterministic synthetic tokens.
This design offers more realistic traffic shapes without exposing the original content. However, AgentX does not test whether a model produces correct code or completes an agent’s task successfully. It measures the serving system, not model intelligence.
The new results used DeepSeek V4 Pro and an early TensorRT-LLM stack. TensorRT-LLM is NVIDIA’s inference runtime for optimizing model execution on its accelerators.
According to the Rubin benchmark analysis, Vera Rubin NVL72 produced about 67 times more total throughput per modeled ownership cost than GB300 NVL72 at 170 tokens per second.
That comparison used TensorRT-LLM and NVFP4, NVIDIA’s four-bit numerical format for lower-precision AI computation. It also measured the full hardware-and-software configuration rather than comparing theoretical chip specifications.
At the same operating point, Rubin achieved a striking advantage because the selected GB300 TensorRT-LLM curve was near its fastest measured endpoint. SemiAnalysis reported a much smaller Rubin advantage against GB300 running SGLang, another serving framework.
The result still matters. Rubin did not win solely because the benchmark selected an outdated generation or a basic single-GPU server. It beat NVIDIA’s Blackwell Ultra rack-scale system while serving the same large model and maintaining a matched response-speed target.
However, the 67x figure describes one point on a performance curve. It does not mean every Rubin deployment generates 67 times more tokens for the same total cost.
At the 60 to 100 token-per-second range, which SemiAnalysis describes as more representative for providers serving this workload, Rubin delivered roughly 1.4 to three times more throughput per total cost.
That is less dramatic than 67x, but it is commercially significant. A sustained twofold gain can reshape capacity planning when power, networking, cooling, and available data-center space already constrain expansion.
The results also show higher maximum interactivity. Rubin reached approximately 276 P90 tokens per second, compared with about 172 for GB300 using the selected TensorRT-LLM configuration.
P90 interactivity measures the streaming speed reached by 90 percent of responses. It helps operators determine whether a throughput result also delivers an acceptable user experience.
Rubin’s advantage was not identical across software stacks. SemiAnalysis found that GB300 running SGLang could reach interactivity similar to Rubin, even though Rubin retained other throughput and latency advantages.
This variation establishes the article’s main tension. NVIDIA Vera Rubin NVL72 appears substantially faster, but its largest reported advantage emerges from a specific combination of model, runtime, precision, and service-level target.
Why Agentic Inference Turns Power Into the Scarce Resource
Rubin’s most consequential gain is not peak arithmetic performance, but the amount of useful agent traffic it can serve inside a fixed power envelope.
An agent consumes more tokens than a simple chat interaction because every step can become context for the next step. Research, coding, and support agents may search databases, run tools, inspect results, and delegate work before answering.
NVIDIA says agentic requests consume about 15 times more tokens than simple chat requests, based on OpenRouter data. That figure will vary by application, but the underlying direction is clear.
As context grows, the system repeatedly processes or retrieves information from earlier turns. Multiple subagents can also create short, intense bursts of concurrent requests.
These characteristics turn memory capacity, memory bandwidth, interconnect latency, cache management, and CPU coordination into first-order constraints. Raw tensor performance remains important, but it no longer explains the complete serving result.
Electricity access adds another limit. An operator may be able to purchase additional accelerators but lack enough grid capacity to power them. New substations, transmission connections, cooling systems, and data halls take longer to build than servers.
Throughput per megawatt therefore measures more than energy efficiency. It approximates how much billable work an operator can extract from a scarce facility resource.
At 100 tokens per second, SemiAnalysis measured Rubin at approximately 59.4 million total tokens per second per utility megawatt. The strongest measured GB300 configuration reached 28.5 million at the same target.
That makes Rubin about 2.09 times faster than the strongest GB300 engine at this practical operating point. GB300 running TensorRT-LLM reached 21.1 million tokens per second per megawatt.
The size of the lead changed as the speed requirement increased. At 150 tokens per second, Rubin retained nearly 37 million tokens per second per megawatt, around 7.2 times the measured GB300 SGLang result.
At 170 tokens per second, Rubin’s throughput-per-megawatt lead was 62.9 times over GB300 TensorRT-LLM. Against GB300 SGLang, however, the multiplier was 5.56 times.
That difference is why every large performance claim needs its engine label. A buyer comparing only accelerator names would miss how much the runtime changes the result.
NVIDIA’s earlier figures showed another view of the same trend. The company reported up to 30 times higher throughput per megawatt than GB300 at 160 tokens per second.
NVIDIA said those tests used the AgentX DeepSeek V4 Pro workload. At publication, the company also said the results were pending SemiAnalysis review and did not include Vera CPU performance for tool calling.
The subsequent review supports a substantial Rubin advantage, but not one fixed multiplier. NVIDIA’s agentic performance data shows the advantage rising from about twofold at 110 tokens per second to 30-fold at 160.
This curve matters more than a single bar chart. An inference provider chooses a balance among concurrency, response speed, time to first token, total latency, and cost.
A configuration optimized for maximum aggregate throughput may make each user wait too long. A system optimized for extreme interactivity may leave expensive capacity underused.
Agentic products add another complication. Tool execution, code compilation, retrieval, and external API calls can leave GPUs waiting for CPUs or remote services.
Rubin addresses this problem with 36 Vera CPUs alongside 72 Rubin GPUs in each NVL72 rack. NVIDIA designed those CPUs to handle agent orchestration, tool calls, data processing, and sandboxed execution.
The economic argument becomes stronger if those components reduce idle time across an entire workflow. It weakens if external tools, network calls, or application logic remain the dominant bottleneck.
For cloud providers and model labs, the pressure is immediate. A large Rubin advantage would make continued Blackwell expansion harder to justify for new power-constrained inference capacity.
Existing Blackwell deployments will not suddenly become uneconomic. Their hardware is already installed, their software is mature, and many workloads do not need Rubin’s highest interactivity range.
The forced response is more selective. Operators must decide which workloads deserve Rubin first, which should stay on Blackwell, and which can move to competing accelerators.
Extreme Co-Design Is the Mechanism Behind the Rubin Gain
NVIDIA’s result comes from coordinating the GPU, CPU, memory, interconnect, runtime, and facility, not from one faster chip working alone.
NVIDIA calls this strategy extreme co-design. Vera Rubin NVL72 integrates 72 Rubin GPUs and 36 Vera CPUs through sixth-generation NVLink inside one rack-scale domain.
The Rubin GPU includes 288 GB of HBM4 memory with 22 TB per second of memory bandwidth. High-bandwidth memory sits close to the processor and supplies model data faster than conventional server memory.
NVLink provides 3.6 TB per second of bandwidth for each GPU and 260 TB per second across the rack. That fabric lets the 72 GPUs behave more like one large computing resource.
This architecture benefits mixture-of-experts models. Such models activate selected expert networks for each token instead of using every parameter for every operation.
Serving them efficiently requires fast routing among processors. Delays moving activations between experts can waste the arithmetic capacity that appears impressive on a specification sheet.
Rubin also improves synchronization operations used during distributed inference. Less communication overhead means processors spend more time computing and less time waiting for coordination.
NVIDIA’s rack architecture combines Rubin with ConnectX-9 networking, BlueField-4 data-processing units, NVLink switches, and the Spectrum-6 Ethernet platform.
The company now describes the broader system as a seven-chip architecture after adding the Groq 3 LPU. An LPU is a processor designed for predictable, low-latency language-model execution.
Rubin handles compute-heavy context processing, also called prefill. LPX racks built with Groq 3 processors can focus on latency-sensitive token generation, called decode.
This separation is known as disaggregated serving. It allows operators to scale prefill and decode resources independently instead of forcing both phases onto identical hardware.
Agent workloads make that split attractive because their requests contain long input histories but may require rapid interactive output. Prefill needs memory capacity and parallel compute, while decode rewards low latency.
Rate matching then balances how quickly the two pools exchange work. Poor matching can leave one group idle while the other becomes overloaded.
The serving software ties these components together. TensorRT-LLM supplies optimized kernels, while NVIDIA Dynamo coordinates inference across distributed resources.
AgentX results demonstrate why software must be treated as part of the product. The gap between GB300 using TensorRT-LLM and GB300 using SGLang reached tens of times at one endpoint.
That does not mean one engine is universally superior. The strongest GB300 engine changed across the tested speed range, with TensorRT-LLM leading at one target and SGLang leading at higher targets.
Rubin’s early result arrived before its software stack had fully matured. That creates room for improvement, but it also introduces deployment uncertainty.
NVIDIA says the platform is in full production and scheduled to ship during the second half of 2026. The company’s earlier platform claim was up to ten times better inference performance per watt than Blackwell.
SemiAnalysis found up to seven times better throughput per megawatt around the operating region compared with Huang’s earlier threefold GTC illustration. At some matched endpoints, the measured multiple moved much higher.
That supports the “sandbagging” interpretation, but only in a narrow sense. Huang’s presentation covered a broad platform expectation, while the new result covers one agentic workload and particular configurations.
Rubin’s design also attacks power provisioning. Data centers commonly size electrical systems for the maximum possible rack draw, even when inference workloads rarely consume that maximum continuously.
NVIDIA DSX MaxLPS uses dynamic power allocation to recover unused headroom across GPUs and racks. It attempts to place more compute within the same site limit without exceeding the facility’s power envelope.
The MaxLPS documentation describes an illustrative one-megawatt inference deployment with 400 GPUs. In that example, dynamic management raises token throughput to 1.35 times the static maximum-power baseline.
NVIDIA says combined facility planning and power control can support up to 40 percent more GPUs in a fixed envelope. That is a planning claim, not a guaranteed result for every site.
Operators must validate their workload’s real power profile, cooling capacity, safety margin, and latency impact. A fleet that frequently reaches peak draw will offer less recoverable headroom.
The mechanism is therefore multiplicative. Faster GPUs, wider memory, lower-latency links, better scheduling, specialized CPUs, and dynamic power management each remove different bottlenecks.
If these layers work together, Rubin can generate more useful tokens from the same building. If one layer fails to scale, the theoretical gain shrinks before it reaches customers.
The 67x Claim Shrinks Outside Its Selected Operating Point
The benchmark supports Rubin’s lead, but it also shows why a maximum performance-per-dollar figure should not become a fleet-wide planning assumption.
The first limitation is the selected endpoint. At 170 tokens per second, the compared GB300 TensorRT-LLM configuration was close to its measured interactivity ceiling.
Small increases in the speed target can sharply reduce the amount of traffic a system serves near that boundary. Rubin still had unused performance range, creating the unusually large ratio.
Comparing Rubin with GB300 SGLang at the same target reduced the throughput-per-megawatt advantage from 62.9 times to 5.56 times. That remains large, but it tells a different purchasing story.
The second limitation is the total-cost model. SemiAnalysis calculates ownership cost using hardware, networking, power, financing, colocation, useful life, and assumed hyperscaler purchasing terms.
A smaller provider will face different financing, utilization, and infrastructure conditions. A cloud renter will also see a different relationship between hardware performance and contracted capacity.
Performance per dollar depends on keeping the equipment busy. An accelerator with excellent peak economics can disappoint if demand arrives unevenly or software prevents high utilization.
The third limitation is workload scope. The published Rubin result focuses on DeepSeek V4 Pro under AgentX’s coding-agent traffic shape.
A customer serving shorter prompts, image generation, retrieval, video, dense models, or offline batch jobs will encounter different bottlenecks. SemiAnalysis itself notes that Rubin’s comparative gains are smaller for offline batch inference and training.
AgentX also uses synthetic payloads. It preserves lengths, shared prefixes, timing, and branching, but it cannot retain the semantic content of private sessions.
Speculative decoding presents a particular challenge. This technique uses a smaller or faster draft model to suggest several future tokens, then asks the main model to accept or reject them.
Synthetic text can produce unrealistic acceptance behavior. AgentX addresses this by using acceptance lengths measured from a separate coding dataset and recording those settings.
That control improves comparability, but it remains an approximation. The benchmark cannot reproduce proprietary provider templates, hidden reasoning, server-side tools, images, or every tokenizer transformation.
Closed-loop execution introduces another nuance. Faster systems progress farther through sampled sessions during the test window, so they can encounter a somewhat different request mix.
None of these issues invalidates the result. They define what the result measures and where buyers should demand additional evidence.
The fourth limitation is early software. SemiAnalysis used a pre-release TensorRT-LLM build, and later releases should improve Rubin’s efficiency.
Early software can also contain regressions, incomplete features, and operational behavior that does not appear in a controlled benchmark. Mature Blackwell stacks have had more time to absorb production fixes.
Reliability matters at rack scale. A complete NVL72 rack contains 1.3 million components and nearly 1,300 chips, according to NVIDIA.
A data-center operator needs sustained goodput, meaning useful output after accounting for failures, maintenance, retries, and unavailable hardware. Peak benchmark throughput does not measure that whole operational record.
The fifth limitation is competitive response. AMD launched its Instinct MI455X accelerator in July 2026 for the Helios rack-scale platform.
The official MI455X specifications list 40.3 petaflops of peak MXFP4 performance and a CDNA5 architecture. Peak arithmetic figures cannot be compared directly with AgentX results.
SemiAnalysis included the prior MI355X in its new measurements. At 100 tokens per second, the strongest tested MI355X configuration reached about 2.01 million tokens per second per megawatt.
Rubin reached 59.4 million at that point, producing a reported 29.5-fold lead. AMD has committed to collaborating on future MI455X AgentX testing, according to SemiAnalysis.
That future comparison will be far more relevant than Rubin against MI355X. It will test two current rack-scale platforms on the same workload, model, traffic shape, and service-level target.
Google’s TPU7x Ironwood also targets large dense and mixture-of-experts models. Its availability through Google Cloud gives customers another route that couples custom silicon with a vertically integrated software platform.
NVIDIA retains an advantage in ecosystem depth. CUDA, TensorRT-LLM, Dynamo, NVLink, and its partner network give the company control over more of the deployment path.
That control can improve optimization, but it can also deepen customer dependence on one vendor. Extreme co-design works best when buyers accept the entire stack.
The credible interpretation is not that Rubin is 67 times better everywhere. It is that Rubin extends the usable inference frontier, especially for large agentic workloads with strict interactivity requirements.
More Tokens per Gigawatt Do Not Automatically Mean More Profit
Rubin can improve data-center economics, but profit depends on utilization, demand, reliability, and selling prices that the benchmark cannot establish.
SemiAnalysis estimates that Rubin can generate more than twice the annual profit per gigawatt of Blackwell. The firm expects that difference to widen as kernels and serving software mature.
That estimate follows a reasonable mechanism. If one megawatt produces more billable tokens, revenue capacity rises while the facility’s grid allocation remains fixed.
Lower unit cost can also expand margins or support lower customer rates. Providers can choose between retaining the efficiency gain and using it to win more demand.
However, throughput represents capacity, not sales. A provider earns more only when customers consume that additional capacity at sustainable rates.
The demand profile must match the hardware. Rubin’s strongest advantages appear in long-context, interactive agent workloads rather than every form of AI computation.
That creates an allocation problem. Providers need enough agentic traffic to keep the new racks busy without diverting workloads that run more economically elsewhere.
Model efficiency can also reduce infrastructure demand per task. Better architectures, shorter reasoning traces, improved caching, and smaller specialized models may lower token consumption.
The opposite effect is equally plausible. Cheaper tokens can encourage developers to build longer-running agents, use more subagents, and automate tasks that were previously uneconomic.
This is the familiar rebound effect of computing. Efficiency lowers the cost of an operation, then software expands to consume the new capacity.
NVIDIA is betting heavily on that outcome. Its framing treats data centers as AI factories whose output is tokens rather than conventional computing services.
The metaphor has limits. Tokens vary widely in value. A token that contributes to a completed coding task is worth more than one generated during a failed reasoning loop.
Agent benchmarks still struggle to connect infrastructure efficiency with completed work. AgentX intentionally holds model behavior outside its scope and does not score output quality.
A full economic test would measure cost per successful task, not only cost per million tokens. It would include model accuracy, retries, tool failures, and the human effort needed to review results.
Latency also affects revenue indirectly. A faster agent can finish more tasks and keep users engaged, but only if the application and external tools respond at comparable speed.
Rubin’s reported end-to-end latency results are encouraging. At one matched ownership-efficiency point, SemiAnalysis measured approximately 20 seconds for Rubin and 60 seconds for B200 or B300.
At a higher throughput-per-cost point, the firm reported about 20 seconds for Rubin and 120 seconds for the compared Blackwell systems.
Those measurements strengthen Rubin’s case because they pair higher throughput with shorter completion times. Still, they remain configuration-specific benchmark results.
Profit also depends on deployment speed. A delayed rack produces no tokens, regardless of its projected efficiency.
NVIDIA designed Rubin around the third-generation MGX rack format to ease installation and service. The compute tray uses a cable-free, hose-free, fanless internal design.
The company says tray assembly and service time fell from nearly two hours to five minutes. Field experience must show whether those design changes improve fleet availability.
Cooling and power density remain serious facility considerations. Rubin concentrates substantial electrical demand into a single rack, requiring compatible liquid cooling and power distribution.
Operators with older facilities cannot capture the gain by exchanging servers alone. They may need new electrical equipment, coolant distribution, networking, and operating procedures.
This makes Rubin most attractive to hyperscalers, model labs, and specialized cloud providers already building new AI campuses. Smaller buyers may access it more efficiently through cloud services.
“The more you buy, the more you earn” works as a memorable summary of NVIDIA’s sales argument. It is not an economic law.
The more accurate version is conditional. The more efficient capacity an operator deploys, fills, powers, and keeps available, the more revenue that fixed site can support.
What Buyers Should Watch After the NVIDIA Vera Rubin NVL72 Result
Three signals will determine whether Rubin’s early benchmark lead becomes a durable production advantage.
The first signal is independently reproducible AgentX data across serving engines. Rubin needs public results from TensorRT-LLM, SGLang, and vLLM using identical models and service-level targets.
That comparison will show how much of the current lead comes from Rubin hardware and how much comes from an unusually favorable software pairing.
Historical results should remain visible as runtimes improve. Otherwise, buyers cannot distinguish genuine hardware gains from software changes that also benefit older systems.
Reproduction by cloud providers would strengthen the case further. Their deployments include scheduling, networking, monitoring, tenancy, and reliability constraints absent from controlled test environments.
If Rubin retains a large advantage across engines and operators, the 67x endpoint will look like an extreme example of a broader shift. If the gap narrows sharply, software selection was the dominant factor.
The second signal is MI455X and TPU7x performance on the same AgentX workload. Current comparisons mix Rubin with accelerators from different product cycles or different benchmark methodologies.
AMD’s Helios platform is the most direct challenger because it combines current GPUs, CPUs, networking, and rack-scale integration. A matched test will reveal whether NVIDIA’s software stack remains its decisive advantage.
TPU7x offers a different competitive route. Google controls the accelerator, compiler, cloud service, and parts of the model stack, giving it its own form of co-design.
If either competitor approaches Rubin’s performance per megawatt at useful interactivity, buyers gain negotiating leverage and more architectural choice. A wide Rubin lead would reinforce NVIDIA’s control of agentic infrastructure.
The third signal is production economics after utilization and reliability. Buyers should track sustained goodput, time to first token, end-to-end latency, power draw, failure rates, and cost per completed task.
They should also separate peak interactivity from the range used by real customers. The 60 to 100 token-per-second region may matter more commercially than an impressive endpoint.
DSX MaxLPS deserves similar scrutiny. Running more GPUs inside a fixed power envelope is valuable only when the system respects site limits without harming latency or equipment life.
A verified 40 percent density gain would multiply Rubin’s hardware advantage. A smaller field result would reduce the projected profit-per-gigawatt improvement.
Developers should care because infrastructure economics eventually shape product design. Lower agent-serving costs can support longer sessions, more retrieval, and more parallel subagents.
Knowledge workers will feel the change indirectly. Faster and cheaper agents can process larger project histories, but valuable output still depends on trustworthy source material.
A well-maintained personal knowledge base can supply that context without treating more generated tokens as a substitute for better evidence.
The early verdict is clear but bounded. NVIDIA Vera Rubin NVL72 has delivered a substantial agentic inference lead, and Jensen Huang’s earlier forecast appears conservative.
The 67x figure is the edge of the curve, not its average. The practical gain sits closer to twofold or threefold in common operating regions, with larger advantages under stricter interactivity targets.
That is still enough to pressure every major AI infrastructure provider. The next question is whether Rubin can preserve those economics after benchmarks become deployments, and tokens must become completed work.
