Noise Out, Bias In: Targeted Bias Injection in Diffusion Language Models via Closed-Loop Activation Steering

AI summary

Overview

Research area: Natural Language Processing / AI safety and security — specifically bias in diffusion language models (dLLMs) and inference-time activation steering.

Technical level: Advanced. The paper assumes familiarity with transformer residual streams, activation steering, masked diffusion decoding, and basic control theory (proportional–integral control).

Scope: The paper introduces and evaluates targeted bias injection, an attack that uses a closed-loop PI controller to steer a frozen masked diffusion language model's answers toward an adversary-chosen demographic group, then compares it against seven fixed-strength steering baselines and four effort-matched ablations across three models and two bias benchmarks.

What This Paper Is About

Masked diffusion language models (dLLMs) generate text by iteratively denoising masked positions, re-predicting each token many times before committing it. An autoregressive model exposes an answer's probability only once, at the step that commits it; a dLLM exposes it at every denoising step, which gives an attacker an observable signal to steer against. The paper asks how much demographic bias an adversary with gray-box hook access to a deployed dLLM's residual stream can inject by using that repeated signal to adjust a steering vector's strength during decoding, without touching weights or prompts.

Key Contributions

1. Identifies the denoising trajectory as a feedback channel. The paper frames dLLM denoising as a control problem where the measured output is the target-answer probability before commitment, and introduces closed-loop targeted bias injection, a decode-time PI controller that adapts one fixed steering direction's strength from that probability while leaving weights and prompts untouched.

2. Demonstrates large bias shifts on LLaDA-8B-Instruct. The attack raises the BBQ target–comparator gap by 14.9 (Black), 22.5 (Arab), and 37.2 (Old) percentage points, and lifts stigmatizing answers on SocialStigmaQA from 17.6% to 58.1%, with no training and about 40 minutes on one GPU per target.

3. Isolates feedback as the cause of the gain. Using a control-theoretic model of the loop plus effort-matched open loops across three dLLMs and seven steering baselines, the authors show feedback wins where no single strength serves every example, and that a tuned constant does better in the one setting where it does.

4. Tests the loop as a defense and audits measurement choices. Running the same controller with a zero setpoint suppresses rather than injects bias, and the paper quantifies how much answer position, parsing rules, and sensor choice change the measured effect.

Main Findings

• Closed-loop steering far exceeds every fixed-strength baseline. On 400 ambiguous Black-target BBQ items under three answer rotations (1,200 prompts), PI lifts the target–comparator gap from 1.8 to 16.7 percentage points. No baseline exceeds 4.6 (open loop, tuned at α=4). The 95% intervals of the three decode-time rows lie entirely above those of every baseline.

• The result does not depend on the calibration items. Removing the 100 items used to fix the gains leaves a gap of 16.2 on the other 300, with PI still ahead of every baseline by at least 12.1 points. Three further item draws give a mean gap of 16.2 against −0.3 for the unsteered model.

• Feedback, not direction or injection site, produces the gain. An open loop matched to the controller's own mean strength (α=3.28) reaches a gap of only 3.5 while corrupting 20.8% of outputs, against 7.8% under feedback. A per-item strength set from the unsteered probe p₀ reaches 4.8; a hindsight oracle holding each item at the controller's own mean reaches 8.4 at the same overall mean of 3.28, and PI leads it by 8.3 points with fewer invalid outputs (7.8% vs 13.4%). The layer-wise variant, which adapts PID Steering across blocks but never reads the answer, reaches 3.1.

• The controller behaves as the mechanism predicts. It raises strength on items that resist and eases off as the target probability nears the setpoint; 78% of sequences reduce their command at least once in response to the measurement.

• The attack transfers across targets and benchmarks. On LLaDA-8B, Arab gives a gap of 22.3 and Old gives Δg = 37.2. On SocialStigmaQA (518 items, 1,554 prompts over seven race stigmas), where directions were fitted on non-race items, PI raises the biased–safe gap by 83.8 points under the strict parser (unsteered gap −44.3), with PI exceeding both open loops. Under a secondary semantic parser, PI's invalidity falls from 17.4% to 1.4% and Δg_sem reaches 99.9, while the fixed α=4 open loop's invalidity rises to 37.8%.

• Transfer across models is mixed. On Dream-7B, a bound of α_max = 1 chosen to keep outputs valid yields Δg = 4.2, a shift whose interval excludes zero although its lead over the baselines does not. On LLaDA-MoE, tuned CAA beats feedback by 2.3 points, a resolved difference; CAA and the matched open loop produce byte-identical outputs there, so this is a genuine win for constant steering.

• The same loop can suppress. With a zero setpoint on Black-target BBQ, it holds the gap at 0.2 with 1.8% invalidity, against 2.2 for the mean-matched constant and 0.8 for AurA suppression. The unsteered gap is only 1.8, so this shows sign-inverted control is stable, not that it reverses a large existing bias.

• The attack has a coherence cost. 7.8% of PI outputs fail the strict parser against 0% unsteered, and every intervention except AurA suppression lowers abstention, which on these ambiguous items is also accuracy.

• Measurement choices matter substantially. The PI gap is 39.5 when the target sits at position A but 5.3 averaged over B and C. A permissive parser lowers the Old open loop's invalidity from 19.0% to 7.5% and raises its gap from 25.8 to 30.8, while PI is unchanged. Adding lowercase letter tokens to the Arab sensor lowers the controller's mean command from 4.56 to 3.60 and changes the gap by −3.8 points, an interval that includes zero.

• Theory bounds when feedback helps. Under the local linear model p_t = γ·p_{t−1} + b·α_t + d for t < τ, a constant strength reaches the setpoint only if α ≥ α*_i = ((1−γ_i)s* − d_i)/b_i, a threshold that belongs to the example. The PI loop is exponentially stable for a range of gains and its integral state converges to α*_i/K_i, whereas proportional control alone settles at a nonzero error.

Methodology in Plain English

The attack has two parts. Offline, the authors fit a single unit direction in the model's residual stream by completing held-out questions twice — once with the target group's option text and once with the comparator's — and taking the normalized mean difference of answer-token activations at block 14. This makes the direction encode the demographic group rather than the display position of the answer.

Online, while the model denoises a response, a proportional–integral controller reads the probability the model assigns to the target answer letter after each forward pass and sets the strength with which the direction is added during the next pass. The direction is added at every position in all 32 frozen blocks. The proportional term responds to the current shortfall from a setpoint, the integral term to its history, so an example below the setpoint gets a growing strength and one that has reached it gets a shrinking one. A clip on the command and an anti-windup guard prevent runaway. The same loop with the setpoint at zero and commands clipped to [−6, 0] acts as a bias suppressor instead of an attacker.

The primary model is LLaDA-8B-Instruct, run frozen with 32-token single-block generation, 64 denoising steps, temperature 0, and low-confidence remasking. Dream-v0-Instruct-7B and LLaDA-MoE-7B-A1B-Instruct test transfer. Primary evaluation uses 400 ambiguous Black-target BBQ questions where abstention is correct, each shown under three rotations of its answer options. SocialStigmaQA is recast with Yes, No, and "Can't tell" options. A strict parser accepts a response only if it begins with an answer letter, and invalid outputs stay in every denominator. Intervals are 95% paired bootstraps, 10,000 resamples, over prompts for BBQ and over templates for SocialStigmaQA.

Why This Matters

Impact on research. The paper argues that a model's bias is a property of the deployed system, not of the checkpoint. It opens a control channel — the denoising trajectory — that is specific to diffusion language models and is invisible to checkpoint audits, prompt-pipeline audits, and to attack methods that fix the intervention before generation. It also extends the control-theoretic framing of activation steering from across network depth to across denoising steps.

Real-world applications:

• Serving-stack audits: providers of dLLM APIs would need to probe the served endpoint rather than the weights, since the attack leaves weights and prompts unchanged.
• Runtime monitoring: the attack leaves a cheap signature on a fixed probe set — lower abstention on ambiguous questions and more invalid outputs — without requiring knowledge of the target group.
• Trusted computing base design: hooks on activations would need to be treated as part of the trusted computing base alongside model weights.
• Fairness evaluation of dLLMs: the results give a concrete stress test for systems considered for hiring and other screening decisions, where the paper notes dLLMs are candidates for the same uses as autoregressive models.

Industry relevance. Any organization deploying open-weight dLLMs behind an API inherits the same forward-hook interface that steering and interpretability libraries use. The paper's threat model assumes a compromised serving-stack component, so the relevant defenses are serving-layer rather than training-layer. The mixed model results also suggest that the vulnerability is not uniform: Dream-7B needed a much tighter strength bound (α_max = 1) to stay coherent, and LLaDA-MoE was not beaten by feedback at all.

Future Directions

• Extending to free-form generation. The current attack reads a multiple-choice answer from a letter probability; free-form generation, where the target has no single token to measure, is untested. The authors name this as a natural next step.
• Testing serving-layer defenses. The paper frames its own zero-setpoint loop as a mitigation but has only tested it where the unsteered gap is already near zero, and it calls for defenses aimed at the serving layer.
• Understanding when constants suffice. The theory predicts feedback pays where the required strength varies widely and adds little where one strength serves every item. LLaDA-MoE and Dream-7B mark two ends of that range, and the paper leaves the precise boundary open.
• Strengthening the control analysis. The control-theoretic model is local and assumes a linear response before commitment; loosening that assumption is an open question.

Target Audience

AI safety and security researchers working on inference-time attacks and activation steering; practitioners responsible for bias audits and serving-stack security for deployed language models; dLLM researchers interested in how the denoising trajectory differs from autoregressive decoding; and fairness researchers who need to understand that measured bias can depend on the deployment stack as much as on the model. The paper carries a warning that it contains examples of stereotyped and stigmatizing content about demographic groups.

Paper details: arXiv:2610.05894v1 [cs.CL], published 05 Oct 2026, licensed CC BY 4.0. Authors are affiliated with Mohamed bin Zayed University of Artificial Intelligence. Code is stated to be publicly available on GitHub.

Authors’ abstract

Masked diffusion language models (dLLMs) generate text by iteratively denoising masked positions, re-predicting each token multiple times before it is committed. An autoregressive decoder exposes an answer's distribution once, at the step that commits it; a dLLM exposes it at every denoising step before commitment, and we show that an adversary can exploit this. Since an answer remains open to revision over many denoising steps, an adversary with access to internal activations can watch how likely the model is to produce a chosen answer and adjust the intervention accordingly. Building on this observation, we study targeted bias injection, an attack that steers a frozen dLLM toward a demographic answer selected by the adversary. The attack uses a simple proportional-integral (PI) controller that tracks the target-answer probability during denoising and adapts the strength of a steering vector on the fly. On ambiguous BBQ questions where the correct answer is abstention, our attack raises LLaDA-8B-Instruct's preference for the targeted group from 1.8 to 16.7 percentage points, more than three times the strongest fixed-strength steering baseline, and on SocialStigmaQA it raises the selection of stigmatizing answers from 17.6% to 58.1%. Fitted to other demographic targets, the same attack shifts answers by up to 37 percentage points, and each attack takes about 40 minutes on one GPU. On the primary target, feedback is what makes the attack work: constant steering at the same average strength over the token-committing steps produces a far smaller shift while corrupting nearly three times as many outputs, and a constant strength set separately for each example still falls well short. Our findings identify the denoising trajectory as a new control channel in dLLMs and call for bias audits that examine the serving stack rather than the frozen model alone.