Skip to content
AI.info

The Pulse

Nvidia claims Vera Rubin inference lead in provisional MLPerf results

Nvidia says Vera Rubin NVL72 delivered over 10 times the throughput per kilowatt of GB300 NVL72 in a Kimi K3 agentic inference test. The Vera Rubin results await MLCommons verification and may change during peer review.

Nvidia claims Vera Rubin inference lead in provisional MLPerf results

Nvidia says its Vera Rubin NVL72 system delivered over 10 times the throughput per kilowatt of provisioned power of its GB300 NVL72 in an agentic inference test using Kimi K3. The comparison, published in an Nvidia technical post on Nov. 16, uses provisional MLPerf Endpoints v1.0 results and a shared minimum responsiveness threshold.

Source developer.nvidia.com

The Vera Rubin results await verification by MLCommons and are subject to change pending peer review. Nvidia says the GB300 results used in the comparison have been verified.

What the benchmark measured

MLPerf Endpoints v1.0 evaluates serving capacity and responsiveness on coding and enterprise workflows accessed through API endpoints. During performance runs, generated tool calls and workflow intent are checked against reference values.

A separate SWE-bench Verified evaluation assesses end-to-end task-solving capability. Nvidia says the checks are intended to test whether faster serving preserves model quality.

Agentic workloads can carry conversation histories across multiple turns, alongside reasoning, tool calls and responses. The benchmark dataset includes coding trajectories that accumulate history and enterprise workflows that reuse instructions, tool definitions and business rules.

The Kimi comparison

In the Kimi K3 comparison, the Vera Rubin run used 32 accelerators and handled a concurrency of 1,024 queries, with system throughput of 41,414.2 tokens per second and end-to-end average interactivity of 50.4 tokens per second per user, according to Nvidia. Its operating-point power was 100.89 kW.

The GB300 run used 48 accelerators at a concurrency of 48 queries. It recorded system throughput of 2,210.4 tokens per second, end-to-end average interactivity of 56.2 tokens per second per user and operating-point power of 60 kW.

The shared minimum for the throughput-per-kilowatt comparison was 50 tokens per second per user of end-to-end average interactivity. For that comparison, Nvidia selected each platform’s best throughput per kilowatt that met the stated minimum; it used a separate matched-concurrency comparison for interactivity.

Results across models

Nvidia also compared the systems on DeepSeek-V4.1-Flash and Qwen3.6-35B-A3B. At the specified responsiveness thresholds, it reports 5 to 10 times higher throughput per provisioned kilowatt across the three models, and 3 to 5 times higher end-to-end interactivity at matched query concurrency.

The company reported a 9-times throughput-per-kilowatt ratio for DeepSeek and a 5-times ratio for Qwen. At matched concurrency, their respective interactivity ratios were 5 times and 3 times.

For Kimi K3, the table gives a 10-times throughput-per-kilowatt ratio and a 3-times interactivity ratio. The table rounds the ratios as shown, while the headline Kimi comparison is described as over 10 times.

Hardware and serving methods

Rubin GPUs combine HBM4 memory, Tensor Cores and sixth-generation NVLink. Nvidia says those components support access to model weights and conversation state, matrix operations and communication across GPUs.

The submissions also use prefix caching, state reuse and Nvidia Dynamo routing. Those techniques are intended to reuse previously processed conversation state and distribute requests across workers, while disaggregated serving allows prompt processing and token generation to be configured independently.

The models have distinct architectures, and the post describes targeted execution techniques including model-specific kernels, optimized memory access and speculative verification. Nvidia says those methods reduce data movement and execution overhead.

Sources

  1. Original source · Blog post NVIDIA Technical Blog developer.nvidia.com

Other pages read

  1. MLCommons endpoints.mlcommons.org

Companies in this story

Related stories

Explore

More articles