BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion

AI summary

Overview

Research area: Natural language processing — efficient conversion of pretrained autoregressive (AR) language models into diffusion language models (DLMs) using parameter-efficient fine-tuning.

Technical level: Intermediate. The paper assumes familiarity with transformer attention, LoRA adapters, and masked-diffusion training objectives, but the core idea is explained clearly enough for readers with general deep-learning background.

Scope: The paper presents and evaluates BYOD, a single-GPU, LoRA-only recipe that converts four instruction-tuned AR model families (Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, Ministral 8B) into masked diffusion generators, comparing each conversion against its frozen AR parent and against LLaDA 8B Instruct.

What This Paper Is About

Diffusion language models can update many tokens in parallel and attend bidirectionally, unlike left-to-right autoregressive models, but leading DLMs have required expensive pretraining from scratch or full-model continued training on hundreds of billions to trillions of tokens. The authors ask whether a cheap, architecture-independent intervention — swapping the causal mask for bidirectional attention and training high-rank LoRA adapters — is enough to induce working diffusion generation in existing AR checkpoints. The goal is a reproducible conversion that runs in hours on a single consumer-cloud GPU, so that diffusion research and cross-architecture comparisons do not require foundation-model-scale compute.

Key Contributions

1. A compute-efficient conversion recipe. BYOD converts four instruction-tuned AR model families to masked diffusion using only LoRA updates (rank 1024 on query and value projections) with every base-model parameter frozen. Each 25,000-iteration conversion takes 3.1–4.0 hours on one NVIDIA RTX PRO 6000 GPU in Google Colab and uses at most 102.4M tokens. The authors state this is the first reported AR-to-DLM conversion of a Mistral model (Ministral 8B Instruct 2410).

2. A paired evaluation design. Each converted model is benchmarked against the exact AR checkpoint it came from, across nine tasks (MMLU, MMLU-Pro, HellaSwag, ARC-Challenge, GSM8K, MATH-500, GPQA-Main, HumanEval, MBPP), plus LLaDA 8B Instruct rerun in the same harness as an external diffusion reference.

3. Controlled analyses of training duration, NFE budget, and adapter merging. The paper reports a 25,000- versus 50,000-update Llama comparison, perplexity and Distinct-1/2/3 across 32/64/128 NFE, validation-loss trajectories, and an ablation folding the LoRA weights into the base projections.

4. A bidirectional attention test and open artifacts. A right-context copy probe verifies functional bidirectionality, and the authors release a standalone training/evaluation implementation plus interactive demos for all four conversions at https://github.com/RuurdKuiper/BYOD.

Main Findings

• Conversion works across all four families. Every converted model generates from fully masked sequences, responds to right-side context, and retains substantial parent-model knowledge, despite no architecture-specific tuning and frozen base weights.

• BYOD-Gemma is the strongest conversion. It reaches 50.6% average accuracy across the nine tasks, compared with 51.9% for LLaDA 8B Instruct in the same harness — a gap of 1.3 points. LLaDA itself scores 2.6 points below its published average in the authors' harness, which the paper uses to illustrate protocol sensitivity.

• AR parents still outperform their conversions on average. Averages in the matched harness: Gemma 65.7%, Qwen 71.6%, Ministral 64.4%, Llama 58.8%, versus 50.6%, 49.0%, 49.9%, and 48.7%/49.2% (25k/50k) for the converted models. Qwen has the strongest AR parent but not the strongest conversion, so parent capability alone does not determine conversion quality.

• The largest deficits are in mathematics and code. On paired HumanEval/MBPP subsets, parsing failed for 28.0% and 8.8% of BYOD-Llama-50k outputs, versus 0% and 0% for Llama AR; the authors attribute this to syntactic errors consistent with parallel token prediction.

• Longer training gives only marginal downstream gains. Extending BYOD-Llama from 25,000 to 50,000 updates raises its average by 0.5 points, with gains on five tasks and declines on four, while validation loss decreases almost monotonically and plateaus toward the end.

• Open-generation fluency is the principal weakness. BYOD perplexity is 8.3–11.1 versus 4.1 for LLaDA and 2.2–3.0 for the AR parents when generating 128 tokens in 64 steps. Across NFE budgets, converted models have 2.7–5.0 times their AR parents' perplexity and generally lower Distinct-n.

• Bidirectionality is confirmed, but clue use varies. Adding a later clue changed the greedy prediction on 100% of test items for every converted model and LLaDA, while exact copying of the visible target was 100% for BYOD-Gemma, BYOD-Ministral, and LLaDA, 90% for BYOD-Llama-25k, 70% for BYOD-Llama-50k, and 20% for BYOD-Qwen. Without the clue, exact-match accuracy was 0% for every model.

• Merging adapters preserves behavior. Folding BYOD-Llama-25k's LoRA into the base projections reduces parameters from 8.466B to 8.030B, slightly improves open-generation metrics (PPL₁₂₈ from 5.081 to 4.764), and lowers the nine-task macro mean by only 0.4 points (48.7% to 48.3%).

• The compute contrast is large but estimated. Under a standardized rental and energy scenario, one BYOD conversion is estimated at 3.2 GPU-hours, $11, and 0.9 kg CO₂e, versus the reported LLaDA run at 130,000 GPU-hours, approximately $674,830, and 44.1 t CO₂e. The authors stress these are illustrative estimates, not their actual bill or measured emissions.

Methodology in Plain English

The recipe has four moving parts.

Attention. In a normal AR transformer, each token can only look at earlier tokens. BYOD replaces the causal mask in every layer with a bidirectional mask, so every prompt, answer, and padding position can attend to every other position. The rest of the architecture and the output vocabulary are untouched.

Adapters. LoRA adapters of rank 1024 are attached to every self-attention query and value projection, with scaling 1024 and zero dropout. Backbone weights, token embeddings, the language-model head, biases, and normalization layers stay frozen. Because the output head is unchanged, no vocabulary expansion is needed: each model reuses an existing single-token string as its mask token (MASK for Gemma, Llama, and Qwen; <?> for Ministral).

Objective. For each example, the instruction is left clean and a masking rate t is sampled uniformly from 10⁻³ to 1. Each answer and EOS-padding position is independently masked with probability t, and the model is trained to reconstruct the clean token at the same position — unlike AR training, there is no label shift. The loss weights answer positions at 0.9 and padding positions at 0.1, with a 1/t correction for the sampled masking rate. The first terminating EOS counts as a genuine answer token; later EOS tokens are padding.

Data and compute. Training uses a one-million-example mixture (45% general instruction following, 18% multiple-choice reasoning, 18% mathematics, 19% code) assembled from Clean-Instruct-3M/Tülu 3/Alpaca-GPT4/Alpaca, MMLU/HellaSwag/ARC-Easy, Orca-Math/GSM8K, and OpenCoder/MBPP, deduplicated and filtered to 256 tokens. The primary runs use 400,000 examples over 25,000 optimizer updates with AdamW at a peak learning rate of 10⁻⁵ under a cosine schedule. Each model gets the same shuffled data, retokenized with its native chat template.

Inference. Generation starts from a fully masked answer block. At each denoising step the model predicts all masked positions, keeps the most confident, and remasks the rest; accepted tokens stay fixed. The number of network function evaluations (NFE) is set independently of output length.

Why This Matters

Impact on research. The paper turns diffusion adaptation from a foundation-model-scale undertaking into a few-GPU-hour experiment. Because the same intervention is applied unchanged to four architectures and each result is paired with its own AR parent, it becomes practical to study how architecture and parent checkpoint affect diffusion generation without re-running pretraining for every comparison. The paper is also candid about the ceiling: cheap conversion does not yet buy cheap, high-quality generation, and the reported gaps in perplexity and code/math accuracy mark a concrete target for the next round of work.

Real-world applications:

• Low-resource and academic labs can run diffusion-language-model experiments on a single rented or free-tier cloud GPU rather than a multi-node cluster.
• Rapid prototyping of editing and infilling interfaces, where bidirectional context and iterative refinement are useful — for example, constrained rewriting or fill-in-the-middle style editing, though the paper's bidirectional test is a synthetic copy probe rather than a product feature.
• Model-family extension for teams that already own an instruction-tuned checkpoint and want to explore diffusion-style generation without abandoning their existing weights or licenses.
• Reproducible benchmarking of diffusion versus autoregressive models under matched data, harness, and parent checkpoints.

Industry relevance. The paper gives a documented cost picture: a conversion is estimated at roughly $11 in rental-equivalent compute versus the $674,830 estimated for the reported LLaDA pretraining run, in the same standardized scenario. For organizations weighing whether to build diffusion capability on top of existing models or train from scratch, that is a useful order-of-magnitude signal — though the authors repeatedly caution that inference quality, not training cost, is the current bottleneck, and that they measure budget in NFE rather than wall-clock latency.

Future Directions

• Post-merge diffusion fine-tuning. The authors propose partial or full diffusion fine-tuning initialized from the merged adapter model, testing whether quality can improve without eroding the capabilities retained from the parent checkpoint.

• What actually drives conversion quality. BYOD-Gemma leads despite Qwen having the stronger parent, and the paper cannot separate architecture, parent quality, model size, and adapter budget because LoRA rank is held fixed. Characterizing the rank–quality–compute trade-off and running controlled architecture comparisons is left open.

• Better training recipes. The flat downstream returns from 25,000 to 50,000 updates suggest longer training alone may be insufficient. The paper names larger effective batches, alternative denoising targets, and adapters in key/output projections, MLPs, or normalization layers as useful ablations.

• Closing the low-NFE generation gap. Perplexity and distinct-n still trail both AR parents and LLaDA, especially when NFE is below output length. The paper calls for inference-quality work and better evaluation than perplexity and n-gram diversity, which do not measure factuality, grammar, or coherence.

Target Audience

Researchers and engineers who already work with transformer language models and want a practical, low-cost entry point into diffusion language modeling; practitioners deciding whether to convert an existing checkpoint rather than pretrain a DLM; and readers interested in parameter-efficient adaptation, benchmarking methodology, or the compute and emissions accounting of language-model training. Readers new to diffusion models will follow the high-level argument but may need background on masked diffusion objectives and LoRA to engage with the method section.

Authors’ abstract

Unlike autoregressive (AR) models, diffusion language models (DLMs) permit parallel token updates, flexible generation, and bidirectional attention, but leading systems rely on costly pretraining or full-model fine-tuning. We present Build Your Own DLM (BYOD), a generalizable LoRA-only recipe for parameter-efficient AR-to-DLM conversion. Without architecture-specific tuning, BYOD induces diffusion generation in Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B; to our knowledge, this is the first reported AR-to-DLM conversion of a Mistral model. The adapters comprise 4.1-5.8% of final parameters while every base-model parameter remains frozen; each 25,000-iteration conversion needs at most 102.4M tokens, processed in 3.1-4.0 hours on one Google Colab GPU. All four models generate from fully masked sequences, use bidirectional context in a controlled attention test, and retain substantial parent-model knowledge. BYOD-Gemma is the strongest BYOD conversion on our nine-task evaluation, reaching 50.6% average accuracy compared with 51.9% for LLaDA 8B Instruct in the same harness. Open-generation quality at low network function evaluation (NFE) budgets remains the principal limitation: BYOD perplexity is 8.3-11.1 versus 4.1 for LLaDA and 2.2-3.0 for the AR parents when generating 128 tokens in 64 steps. To enable further experimentation and inspection, we provide a standalone training and evaluation implementation and interactive demos for all four conversions. BYOD thus provides a practical route for extending diffusion generation to other existing model families while identifying the gap that efficient conversion must next close.