The Pulse
Apple Details DACA-GRPO for Diffusion Language Models
Apple's September 2026 research page describes DACA-GRPO, a reinforcement-learning method for diffusion language models. No attributable quotation appears on the page; the source contains a technical summary and author list only.
AI.info Team ·
Apple targets two weaknesses in diffusion-model training
Apple researchers have proposed a reinforcement-learning method designed to improve how diffusion language models learn from rewards. The technique, called Denoising-Aware Credit Assignment for GRPO, or DACA-GRPO, focuses on the sequence of intermediate denoising decisions that gradually turn corrupted tokens into a finished response.
The work addresses two problems identified by the authors. Existing reinforcement-learning methods for diffusion language models treat every denoising step as equally important, even though some steps have a greater effect on the final output. They also rely on mean-field likelihood estimates that the researchers describe as systematically biased and high variance.
Apple published the research in September 2026. The paper was authored by Amin Karimi Monsefi, Dominic Culver, Nikhil Bhendawade, Lokesh Boominathan, Manuel R. Ciosici, Yizhe Zhang and Irina Belousova. Apple’s research page identifies the work as a methods-and-algorithms paper covering speech and natural-language processing.
DACA-GRPO assigns credit inside the denoising trajectory
DACA-GRPO adds two mechanisms to GRPO-style trainers. The first, called Denoising Progress Scores, extracts per-token importance weights from intermediate predictions. The researchers say the scores require no additional forward pass, allowing the trainer to estimate which tokens and denoising stages contributed most to the eventual result.
The second mechanism, Stratified Masking Likelihood, divides token positions into groups, or strata. Each token is then evaluated with most of the sequence available as context. The design is intended to reduce the bias introduced when likelihood calculations approximate the full sequence using a mean-field estimate.
That combination changes the training signal without requiring a new model architecture. Apple describes DACA-GRPO as a lightweight enhancement that can be applied to any GRPO-style trainer, making the proposal a change to post-training rather than a replacement for diffusion-language-model pretraining.
Results span reasoning, code and structured output
The researchers apply DACA-GRPO on top of three GRPO base methods and evaluate the resulting systems across seven benchmarks. The tests cover mathematical reasoning, code generation, constraint satisfaction and constrained generation.
Apple reports maximum gains of 5.6 percentage points on math-reasoning tasks and 7.4 points on code-generation tasks. The largest improvement comes in constraint satisfaction, where the reported gain reaches 36.3 points. The method also improves adherence to JSON schemas by as much as 5.9 points.
The figures are reported as improvements across the benchmark suite rather than as a single overall score. Apple’s summary does not identify one model size or one training configuration as responsible for every maximum gain, so the headline results should be read as the best improvements observed among the reported experiments.
Why intermediate steps matter for diffusion language
Autoregressive language models produce text one token at a time, which gives many conventional reinforcement-learning methods a relatively direct sequence of decisions to optimize. Diffusion language models take a different route: they begin with a heavily corrupted or masked representation and repeatedly refine many positions over a denoising trajectory.
A final reward, such as a score for a correct solution or valid code, does not automatically reveal which intermediate decisions produced it. Treating all stages alike can therefore send the same learning signal to steps that shaped the answer and steps that merely cleaned up an already coherent sequence.
DACA-GRPO attempts to make that distinction explicit. Its progress scores estimate token importance during refinement, while stratified likelihood calculations aim to give the policy optimizer a less distorted view of the model’s probabilities. The approach preserves the group-relative structure of GRPO while adding information about when and where the output took shape.
A research direction for non-autoregressive language models
Diffusion language models have attracted interest because they can revise multiple positions in a sequence rather than committing to a strictly left-to-right generation path.
DACA-GRPO addresses a narrower problem: how to apply reinforcement learning after a diffusion language model has been trained. Its reported results suggest that the denoising trajectory contains useful training information that standard GRPO implementations leave unused. The paper does not establish that the method will improve every diffusion architecture or task, but it provides a way to test trajectory-aware credit assignment without rebuilding the trainer from the ground up.
The primary technical description is available from Apple Machine Learning Research, with the full paper available through arXiv.