DSB: Dynamic Sliding Block Scheduling for Diffusion LLMs

AI summary

Overview

Research area: Natural Language Processing — specifically efficient inference and decoding for diffusion large language models (dLLMs).

Technical level: Intermediate. The algorithms are conceptually simple, but the paper assumes familiarity with masked diffusion decoding, block-diffusion (semi-autoregressive) inference, and KV-cache mechanics.

Scope: The paper proposes a training-free block scheduling method (Dynamic Sliding Block, DSB) and a companion training-free KV-cache (DSB Cache) that together improve the quality–speed trade-off of block-diffusion LLM inference, evaluated across four model variants and five benchmarks.

Authors are affiliated with Nanyang Technological University (Lizhuo Luo, Shenggui Li, Yonggang Wen, Tianwei Zhang), with equal contribution noted. The arXiv paper is 2602.05992v3 [cs.CL], published 2026-02-05 under an arXiv.org perpetual non-exclusive license, and code is released at https://github.com/lizhuo-luo/DSB.

What This Paper Is About

Diffusion large language models decode many positions in parallel, but decoding globally can misalign generation order with the causal structure of language. The standard fix is block-diffusion inference, where the response is split into fixed, predefined blocks decoded left-to-right. That fixed partitioning is blind to how difficult or ambiguous each position actually is: it can force premature commitments to low-confidence tokens inside the active block while blocking already-confident tokens just outside it. The paper's goal is a block schedule that adapts online to semantic difficulty, plus a KV cache that survives the block movement, all without any training or fine-tuning.

Key Contributions

1. Dynamic Sliding Block (DSB): A training-free decoding schedule where the active block slides forward and changes size across denoising steps. Its left boundary advances to the first still-masked position inside the block (or to the block's right edge if none exists), and its right boundary expands to keep at least an initial number of unresolved tokens in view, capped at a maximum block size.

2. DSB Cache: A training-free KV-cache mechanism tailored to sliding blocks. It caches positions outside the active block and a prefix window immediately preceding it, refreshing the prefix window and active block at every step while periodically performing a global refresh.

3. Diagnosis of cache instability: The paper identifies why existing fixed-block caching schemes (prefix caching, dual caching) degrade under sliding blocks — newly exposed boundary positions have transient rather than stable KV states, causing frequent invalidation and recomputation.

4. Two practical variants and broad evaluation: DSB (const.), where the maximum block size equals the initial size (sliding without resizing), and DSB (greedy), where the maximum is unbounded. Both are evaluated across LLaDA-8B-Instruct, LLaDA-1.5, Dream-v0-Base-7B, and Dream-v0-Instruct-7B on GSM8K, MATH, HumanEval, MBPP, and BBH, with ablations on block lengths, generation length, and prefix-window length.

Main Findings

• Naive block scheduling is a bottleneck: Because a fixed block is agnostic to semantic difficulty, it forces commitments to low-confidence positions inside the active block and delays high-confidence positions near the boundary. The paper's Figure 1 example shows naive scheduling decoding "six" instead of the ground-truth "zero" at later steps.

• DSB improves the quality–speed trade-off without caching: On LLaDA-8B-Instruct under confidence-aware parallel decoding, DSB (greedy) reaches 78.54 accuracy on GSM8K with 51.03 TPS. On HumanEval, DSB (const.) achieves 42.07 accuracy and 124.6 TPS, exceeding AdaBlock-dLLM by 2.44 accuracy points and 14.0 TPS.

• DSB Cache amplifies the gains: With DSB Cache, DSB (greedy) reaches 80.29 and 81.96 accuracy on GSM8K for the two LLaDA variants, both above vanilla sampling, at 99.61 and 95.54 TPS — versus 92.26 and 86.42 TPS for naive block with Dual Cache.

• Dream results are less consistent without a cache: On Dream-v0-Base-7B, DSB (const.) achieves 34.70 accuracy and 72.05 TPS on MATH, and 52.44 accuracy and 78.09 TPS on HumanEval with the base model. On Dream-v0-Instruct-7B, DSB (greedy) with cache gets 73.08 accuracy and 75.27 TPS on GSM8K, beating naive block with dKV-Cache-Decode by 1.21 accuracy points and 33.15 TPS. The paper attributes weaker uncached Dream gains to that model's AR-initialized, shift-aligned training.

• The prefix window in DSB Cache is essential: Ablating it and using Dual Cache instead drops LLaDA-8B-Instruct from 80.14 to 76.42 accuracy and from 98.10 to 78.93 TPS on GSM8K for DSB (const.); HumanEval falls from 37.80 to 28.05 accuracy and from 105.3 to 87.79 TPS.

• DSB is robust to initial block length: Across tested initial block lengths on GSM8K with LLaDA-8B-Instruct, DSB consistently improves throughput and improves accuracy in most cases, except at S_init = 64, where the block reaches farther positions earlier, weakening causality.

• Maximum block size trades quality for speed: On LLaDA, increasing S_max raises TPS but can hurt accuracy. On Dream, TPS gradually decreases as S_max increases, while accuracy rises then falls, with S_max = 64 a clear turning point.

• Minimum prefix-window length has a turning point: For LLaDA, accuracy and throughput both rise then fall as the window grows, with ℓ_pmin = 24 a clear turning point; for Dream, accuracy decreases slightly and throughput generally decreases as ℓ_pmin increases. Defaults are 24 for LLaDA and 4 for Dream.

Methodology in Plain English

Diffusion LLMs start with a fully masked response and fill in positions over many denoising steps. The paper keeps that iterative unmasking but replaces the rigid block partition with a moving window:

• Sliding left edge. After each round of unmasking, the method scans the active block for the first position still masked and sets the left edge just before it. If everything in the block is resolved, the left edge jumps to the right edge.
• Adaptive right edge. The right edge expands so the block keeps roughly the initial number of unresolved tokens, but never grows beyond a maximum size.
• Result. Easy, high-confidence positions near the boundary get decoded sooner; uncertain positions inside the block wait for more reliable context instead of being committed prematurely.

For caching, the key insight is that when the block slides, the tokens it just exposed have KV states that are still changing. Caching them as if they were stable causes repeated invalidation. So the method keeps a small "prefix window" immediately before the active block, refreshes that window and the active block every step, caches everything else, and does a full global refresh periodically (after decoding S_init tokens) to re-synchronize.

Experimental setup: All experiments used an NVIDIA H200 140G GPU with the official lm-eval library. Generation length was 256; block length and S_init were 32; the confidence threshold for parallel decoding was 0.9. Benchmarks were GSM8K (5-shot), MATH (4-shot), HumanEval (0-shot), MBPP (3-shot), and BBH (3-shot). Baselines spanned decoding (vanilla Top-1 and Fast-dLLM's confidence-aware parallel decoding), block scheduling (Naive Block and AdaBlock-dLLM), and caching (Dual Cache and dKV-Cache-Decode). Metrics were accuracy for quality and tokens per second (TPS) for speed.

Why This Matters

Impact on research: The paper argues that block-diffusion inference has been limited by a fixed schedule that ignores semantic difficulty, and shows that a training-free, online-adaptive schedule plus a matching cache can push the quality–speed frontier further. It positions itself against delimiter-driven scheduling (AdaBlock-dLLM) and diffusion-expansion approaches (WavefrontDiffusion), and distinguishes its training-free design from the concurrent training-based WeDLM. Because it requires no retraining, the approach can be applied on top of existing block-diffusion models.

Real-world applications (mapped to the evaluated tasks):

• Reasoning and math assistants, where the paper evaluates on GSM8K and MATH.
• Code generation and completion tools, evaluated on HumanEval and MBPP.
• General-purpose instruction-following and multi-task reasoning assistants, evaluated on BBH.
• Deployment scenarios where serving cost and latency matter, since TPS is the paper's efficiency metric and the method is training-free.

Industry relevance: Faster decoding at equal or better accuracy directly reduces serving cost per token for any product built on diffusion LLMs. Because DSB and DSB Cache require no training and work with existing KV-cache infrastructure, they are drop-in candidates for inference stacks — the paper's gains appear in the cached setting, which it identifies as the most widely used deployment configuration.

Future Directions

• Reconciling the Dream model results. Performance on the Dream model without a KV cache is inconsistent, which the paper links to its AR-initialized, shift-aligned training. Understanding and closing that gap is an open question.
• Settling the maximum block size choice. The effect of S_max varies across datasets and model architectures; the paper recommends the simpler Const. and Greedy variants, leaving a more principled rule for setting the cap unexamined.
• Pushing the adaptive schedule further. DSB abandons handcrafted delimiter heuristics, but whether learned (rather than rule-based) boundary policies could do better is not explored.
• Extending beyond the tested settings. Evaluation covers four model variants and five benchmarks at generation length 256 in the main tables; the paper reports generation-length ablations only on HumanEval and notes a degradation on LLaDA at generation length 1024. Broader length and domain coverage remains open.

Target Audience

Researchers and engineers working on efficient LLM inference, particularly diffusion and masked-diffusion language models. It will be most useful to practitioners who already understand semi-autoregressive block decoding and KV caching and who want a training-free way to improve both quality and throughput, as well as to researchers studying alternative decoding schedules and their interaction with cache stability.

Authors’ abstract

Diffusion large language models (dLLMs) have emerged as a promising alternative for text generation, distinguished by their native support for parallel decoding. In practice, block inference is crucial for avoiding order misalignment in global bidirectional decoding and improving output quality. However, the widely-used fixed, predefined block (naive) schedule is agnostic to semantic difficulty, making it a suboptimal strategy for both quality and efficiency: it can force premature commitments to uncertain positions while delaying easy positions near block boundaries. In this work, we analyze the limitations of naive block scheduling and disclose the importance of dynamically adapting the schedule to semantic difficulty for reliable and efficient inference. Motivated by this, we propose Dynamic Sliding Block (DSB), a training-free block scheduling method that uses a sliding block with a dynamic size to overcome the rigidity of the naive block. To further improve efficiency, we introduce DSB Cache, a training-free KV-cache mechanism tailored to DSB. Extensive experiments across multiple models and benchmarks demonstrate that DSB, together with DSB Cache, consistently improves both generation quality and inference efficiency for dLLMs. Code is released at https://github.com/lizhuo-luo/DSB.