Multimodal Model Diffing for Feature Discovery and Control

AI summary

Overview

Research area: Mechanistic interpretability of Multimodal Large Language Models (MLLMs), combining sparse autoencoders (SAEs), model diffing, and activation steering. Published under Computer Vision on arXiv (2608.09928v1, 10 Aug 2026) by authors affiliated with the University of Oxford and Microsoft.

Technical level: Advanced.

Scope: The paper introduces MMDiff, a framework that trains multimodal SAEs on top of base-LM SAE dictionaries, diffs the two to isolate features reshaped by multimodal training, discovers task-specific feature subsets via per-token contrastive firing analysis, and uses those features for causal removal and targeted steering across spatial reasoning, multimodal safety, and OCR tasks.

What This Paper Is About

MLLMs can read text in images, localize objects, and reason about spatial relations, but it is unclear which internal features produce these behaviors, which makes auditing failures or suppressing unsafe outputs difficult without retraining. Since SAEs trained directly on MLLM activations mix features inherited from the language backbone with features actually altered by multimodal training, the authors propose diffing a base-LM SAE against its multimodal-adapted counterpart to separate the two. The goal is to turn the resulting feature differences into concrete interfaces that both discover task-relevant features and allow targeted control over model behavior.

Key Contributions

1. Multimodal model diffing. The authors train multimodal SAEs initialized from base-LM dictionaries and identify features that are geometrically reshaped and visually responsive after multimodal adaptation.
2. Task-specific feature discovery. They introduce a per-token contrastive firing pipeline for finding task-relevant features, demonstrated on visual-spatial understanding, multimodal safety, and OCR.
3. Feature-level control. They show that MMDiff-discovered features act as intervention handles: feature-level causal removal suppresses target behaviors such as unsafe responses, while MMDiff-CAA steering improves spatial reasoning and OCR.
4. Supporting analyses. Attribution patching is used to localize task-specific attention heads, and automated interpretation is used to label discovered features with natural-language descriptions.

Main Findings

• Spatial feature removal degrades spatial accuracy selectively. Ablating top spatial features lowers VSR accuracy by 6–31%, with means of −10.1%, −12.3% and −14.6% for MMDiff-Llama, MMDiff-Gemma and MMDiff-Qwen, while leaving general VQA nearly unchanged (|ΔVQA| ≤ 1.5%) and control deltas near zero.
• Instruction tuning creates the spatial behavior. Ablating the same MMDiff-Gemma features on the pretrained PaliGemma 2 variant (pt-448) versus the instruction-tuned variant (mix-448) shows instruction tuning amplifies the causal contribution by roughly 3x on average. Two features (L13/F15219, L12/F2257) reverse sign, acting as noise before instruction tuning.
• Steering beats a single-layer baseline. MMDiff-CAA improves ΔVSR by +3.6% on average over vanilla CAA across ten spatial features on PaliGemma 2 (peak +15.4% on "ahead of"), and improves OCR by +1.8% on average across five OCR features (peak +10.58% on L17/F13602).
• Decomposition of the steering gain. On the same spatial features, single-layer CAA gives +8.96, extending CAA to the discovered feature layers gives +10.78, and adding the feature's decoder direction gives +12.59.
• Safety features reduce attack success. Each per-category top feature on PaliGemma 2 reduces VLSBench ASR by 17–28% with |ΔVQA| ≤ 1% and ΔCtrl ≤ 1%. Across a sweep of 1,061 candidate safety features, the mean effect is ΔASR = −9.67%, ΔVQA Acc = −0.03%, and ΔCtrl = +0.41%.
• OCR removal is targeted. Across five top features the mean ΔCat is −16.9% with |ΔVQA| ≤ 1.6% and |ΔCtrl| ≤ 1.8%.
• The adapted-feature filter is necessary. Dropping it produces larger VSR drops but degrades general VQA by 24–26%. The adapted-feature filter alone leaves VQA intact but yields almost no task effect (−1.0% VSR, −0.2% VQA). Only the full pipeline is both causally effective and selective (−12.3% VSR, −0.1% VQA). Ablating randomly-selected features from the same layers moves VSR by −0.5%.
• Diffing is not replaceable by standard MLLM SAE training. A randomly initialized SAE trained on LLaVA-MORE activations with identical data and hyperparameters degenerates: its top 10 spatial features by odds ratio all lie in a single early layer and fire on 100% of VSR samples, and ablating them leaves VSR unchanged (mean +0.22, no feature beyond ±1.4), against −10.11 for MMDiff features on the same model.
• Modality masking matters. Text-only SAEs achieve the lowest FVU and remain most aligned with the base-LM dictionary, while image-only and full-sequence variants show larger early-layer rotations due to the projector-induced distributional gap.
• Adapted sets are sparse. Adapted features constitute roughly 5%, 20% and 13% of all features for MMDiff-Llama, MMDiff-Gemma and MMDiff-Qwen.
• Head-level localization. Attribution patching curves for spatially selective features typically peak in middle layers, and a small subset of heads scores notably high, with the same heads recurring across related spatial relations (for example, head L13H1 for queries about "on top of").

Methodology in Plain English

The researchers start with sparse autoencoders that already decompose a base language model's hidden states into interpretable feature directions. They then fine-tune these dictionaries on multimodal activations from the corresponding vision-language model so the features align with the vision-language space, training each SAE on cached activations from 50k VQAv2 image–question pairs and comparing masked variants (full-sequence, image-only, text-only) using fraction of variance unexplained on a held-out split.

To find features that multimodal training actually changed, they apply two filters. First, they compare each feature's decoder direction before and after adaptation using cosine similarity, keeping features in the bottom 25% (strong rotation). Second, they measure each feature's mean squared activation under vision inputs, discarding inactive directions with a simple cutoff since nearly half of features never fire on vision. Together these give the "adapted" set.

Within that set, they look for task-specific features by counting how often each feature fires per token on a target dataset versus a generic VQAv2 baseline, retaining features with an odds ratio of at least 3 and a frequency gap of at least 0.05, screened by a Fisher exact test with Benjamini–Hochberg correction. To rule out prompt-lexical artifacts, they rewrite the questions with neutral prompts such as "Describe how the items are arranged." and keep only features that still fire.

For control, causal removal projects out a feature's decoder direction at every transformer layer and only at text-token positions, applied at the attention-block output, the MLP-block output, and the layer residual output. Steering (MMDiff-CAA) combines multi-layer backbone contrastive activation steering at detected top feature layers with injection of a discovered feature's decoder direction at its feature-associated layer.

Why This Matters

Impact on research. The paper shows that the difference between a base-LM SAE and its multimodal-adapted counterpart is itself a usable discovery signal, rather than something to be treated as noise. It extends feature-level model diffing from the language-only setting to the language-to-multimodal transition, and demonstrates that multimodal SAEs can function as control mechanisms rather than only interpretability tools. The cross-stage ablation on PaliGemma 2 provides evidence that specific spatial behaviors are introduced during multimodal instruction tuning rather than inherited from pretraining.

Real-world applications (from the behaviors studied):

• Safety auditing and filtering of multimodal assistants, where harmful intent is only inferable from the image and text together, using feature-level removal to reduce attack success rate without degrading general visual question answering.
• OCR-dependent pipelines such as document processing or accessibility tooling, where feature-level control can steer or selectively suppress scene-text reading behavior.
• Spatial reasoning assistants used in robotics, navigation, or embodied agents, where steering discovered features improved spatial accuracy in the reported experiments.
• Model debugging and compliance review, since auto-interpretation produces natural-language labels for features alongside contrastive-firing metrics.

Industry relevance. The results suggest that model providers can build feature-level interfaces for auditing, steering, and controlling MLLM behavior without retraining the underlying model, which is directly relevant for deploying multimodal systems under safety constraints. The reported absence of VQA spillover is central to that claim, since it indicates targeted intervention rather than general capability degradation.

Future Directions

• Broader architecture coverage. MMDiff is instantiated on three backbones (LLaMA-3.1-8B, Gemma-2-2B, and Qwen3-1.7B). Applying the full recipe to additional MLLM families, including larger backbones, mixture-of-experts variants, and Qwen-VL or Pixtral-style architectures, is identified as a natural next step.
• Extending safety and OCR evaluation beyond one model. Safety and OCR are evaluated on PaliGemma 2 only, so generalizing those findings to the other MLLM families remains open.
• Reducing dependence on a reference model. MMDiff-CAA assumes access to an instruction-tuned reference model from which the steering direction can be extracted; where only the base or only the instruction-tuned model is available, the recipe reduces to standard SAE-feature steering.
• Handling failure modes in safety ablation. A minority of safety candidates cause generation collapse rather than refusal when ablated and currently require a post-hoc filter to exclude, which raises the question of how to identify such features before intervention.

Target Audience

Researchers and engineers working on mechanistic interpretability, sparse autoencoders, and multimodal large language models, particularly those interested in feature-level auditing and activation steering for vision-language systems. It is also relevant to safety teams evaluating multimodal model behavior and to practitioners seeking targeted control methods that avoid degrading general visual question answering. The paper assumes familiarity with SAEs, residual-stream interventions, and multimodal architectures, so it is best suited to readers with an advanced background in model internals.

Authors’ abstract

Multimodal Large Language Models (MLLMs) exhibit strong visual understanding, yet the internal features that cause these behaviors remain difficult to identify, audit, or control. While applicable to post-hoc inspection, hidden states that are decomposed into interpretable feature directions using sparse autoencoders (SAEs) neither readily isolate which features are changed by multimodal training, nor are they directly useful for targeted control. We introduce MMDiff, a multimodal model-diffing framework that trains multimodal SAEs and turns them into feature-level interfaces for discovering and controlling multimodal behavior. MMDiff supports three uses: (i) feature isolation, by diffing a base-LM SAE against its multimodal-adapted counterpart to identify features altered by multimodal training; (ii) task-specific feature detection, via per-token contrastive firing analysis that isolates causal features; and (iii) feature-level control, by causally removing or steering the discovered feature directions. We train multimodal SAEs for three MLLM families, LLaVA-MORE, PaliGemma 2, and InternVL3.5, and evaluate on visual-spatial understanding, multimodal safety, and OCR. MMDiff discovers sparse, causally specific features whose removal selectively degrades target behaviors by an average of 12% on spatial tasks and 17% on OCR, and reduces attack success rate by 24% on multimodal safety attacks, with no impact on VQA performance. Steering these features improves spatial and OCR accuracy by +3.6% and +1.8% on average over a standard single-layer steering baseline. These results show that multimodal SAEs can serve not only as interpretability tools, but as mechanisms for auditing, steering, and controlling MLLMs behavior toward safer and more capable generations.