The Pulse
MLPerf Training adds agentic and recommender tests in v6.1
MLPerf Training v6.1 adds a post-training test for software engineering tasks and replaces its recommendation benchmark. The results cover 153 systems submitted by 24 organizations.

New training workloads
MLPerf Training v6.1 adds a test of agentic language-model post-training and replaces its recommendation benchmark with one built around users’ sequences of interactions. MLCommons published the results on November 16.
The round drew submissions from 24 organizations, covering 153 systems and producing 250 test results. The suite measures how quickly systems train models to a specified quality target, testing hardware and software together.
Teaching a model to repair code
The new language-model test uses Qwen3.5-397B and a post-training method called Group Relative Policy Optimization, or GRPO. Its model is a mixture-of-experts system with 397 billion total parameters and 17 billion active parameters per token.
The benchmark uses the R2E-Gym dataset of software engineering problems. It selects a subset with 700 training problems and a separate set of 251 validation problems.
The agent receives a software issue and a container environment containing a repository and its dependencies; its task is to produce a patch. Each attempt receives a pass-or-fail reward, and GRPO uses those results to update the model.
The target is a pass@4 score of 0.69: the model must solve 69% of validation problems at least once in four attempts. Evaluation uses test files the agent cannot see or modify, a measure intended to prevent it from obtaining a reward by changing tests instead of fixing a problem.
A new recommendation test
The recommender test, DLRMv4-HSTU, replaces DLRM-dcnv2 from earlier versions of the suite. The new test uses an attention-based architecture that reads a user’s interaction history as a sequence.
Its Yambda-5B dataset contains music listening activity released by Yandex Music. The dataset contains approximately 4.79 billion interactions, 1 million users and 9.39 million items, according to MLCommons.
The quality target for DLRMv4-HSTU is 0.75 AUC. The reference implementation turns the activity log into user interaction sequences and adds feature tables to the dataset.
Systems in the results
MLCommons said 57 of the submitted systems, or 37% of the total, were cloud-based, up from 29% in v6.0. Twelve systems used 1,000 or more accelerators, and the largest used 10,240.
The hardware included NVIDIA, AMD and Google accelerators, with the first MLPerf Training results for NVIDIA Rubin-generation GPUs and Google TPUv7, known as Ironwood. Crusoe was a first-time submitter to the training suite.
The release also includes visualization tools for comparing performance across systems and ranking results for systems of similar scale. The closed division requires the same model as the reference implementation, while the open division permits a different model or retraining.