Reversion, Retention, and Semantic Intervention: Measuring Functional Content in CLIP Fine-Tuning Deltas

AI summary

Overview

Research area: Weight-space interpretability and model auditing for fine-tuned vision–language models (CLIP), combining semantic readouts of weight deltas with causal reversion/retention interventions.

Technical level: Intermediate to Advanced. The paper assumes familiarity with CLIP, vision transformers, attention value–output decomposition, singular value decomposition, and fine-tuning; the prose is clear but the methodology is dense.

Scope (one sentence): The paper measures how much of a CLIP model's fine-tuning improvement is actually carried by specific parameter groups (attention VO, QK, MLP, LayerNorm) and by semantically identified weight directions, using paired reversion, retention, and matched-random controls across five classification settings.

What This Paper Is About

Fine-tuning changes a pretrained model's weights, and prior work can attach semantic labels to those changes (the "fine-tuning delta"), but it is unclear how much of the new task performance actually depends on the labeled updates. A small accuracy drop after removing an update does not prove that update was functionally unimportant, because other updates in the same network can compensate. The authors address this by measuring both directions of the question: what happens when you remove selected updates (reversion), and how much improvement comes back when you keep only selected updates (retention).

Key Contributions

1. Paired reversion and retention across three metrics. The authors measure the functional role of fine-tuning updates using accuracy, cross-entropy loss, and correct-class score margin, rather than accuracy alone, showing that the apparent contribution of a component depends on the metric chosen.

2. Specificity and overlap tests for retained recovery. They compare structured retention against ten per-layer, parameter-count-matched random delta subsets (seeds 7000–7009), and quantify shared corrections of the pretrained model's errors to characterize what they call operational redundancy.

3. Semantic readout validation through anchor substitution and direction removal. They test whether task-aligned directions found by a SITH-style semantic readout actually matter, using a null tune with generic text anchors and readout-guided removal versus matched-random removal inside VO-retained models.

4. A practical evaluation protocol. They propose a sequence for auditing weight edits: specify the parameter and bias mask, evaluate base and tuned models on identical examples, pair each reversion with a defined retention, report raw performance plus normalized recovery plus a continuous metric, and include matched-random and shared-correction controls.

Main Findings

• Low necessity coexists with high retention. On synthetic financial charts, accuracy rises from 0.1359 to 0.9846 after tuning, yet reverting the attention VO updates leaves accuracy unchanged at 0.9846 (necessity cost 0.000, 95% CI [-0.008, 0.008]). Retaining VO alone recovers 0.808 of the improvement (retained model accuracy 0.8218). MLP-only retention recovers 0.989.

• Retention ranges across settings. VO-only accuracy recovery spans 0.428–0.869 (Charts 0.808, Flowers102 0.547, EuroSAT 0.869, Aircraft 0.428, Flowers102 ViT-B/16 0.438); MLP-only recovery spans 0.682–0.989 and is higher than VO in every setting. QK-only recovery is lower, 0.132–0.354.

• Necessity costs are mostly small but not uniformly so. Twelve of fifteen component-setting cells have point estimates at most 0.05. Exceptions include Aircraft VO at 0.0605, and Aircraft and ViT-B/16 MLP reverts at 0.1270 and 0.1240. Flowers102 MLP is 0.0495, with an interval spanning the threshold.

• Loss preserves the contrast; margins do not. On Charts and EuroSAT the VO-revert loss cost intervals span zero (-0.0081 [-0.0245, 0.0055] and 0.0035 [-0.0154, 0.0211]) while VO retention recovers 0.746 and 0.822 of the loss improvement. But VO reversion lowers mean correct-class margin in all five settings (by 1.519, 1.458, 2.940, 0.981, and 1.363); on Charts the margin falls from 7.576 to 6.057 while accuracy stays at 98.46%. VO-only margin recovery is only 0.225–0.283 versus 0.470–0.687 for MLP.

• Retained recovery is component-specific under count matching. Weights-only VO retention beats all ten random masks on loss recovery in every setting (e.g., Charts 0.745 vs. random mean 0.555, range [0.554, 0.556]; EuroSAT 0.820 vs. 0.609, range [0.608, 0.611]). The advantage over the mean random mask is roughly 0.19–0.29 of the loss gain and persists on unsaturated Aircraft, where weights-only VO reaches 0.4930 accuracy versus at most 0.4035 for random masks.

• Retentions correct overlapping errors. The fraction of pretrained errors corrected by both VO and MLP retention is 0.7893, 0.5211, 0.8143, 0.2502, and 0.3977 across the five settings, exceeding the product of the marginal correction rates by 0.018 to 0.114 with intervals excluding zero.

• Readout-guided removal beats random removal, but the strict criterion mostly fails. The identified-minus-random collapse difference is positive in all five settings (Charts 0.366, Flowers102 0.081, EuroSAT 0.736, Aircraft 0.102, ViT-B/16 0.177), with random-control ranges at most 0.009. However, only EuroSAT meets the pre-specified majority-contribution criterion requiring removal to eliminate at least half of VO-retained recovery and at least twice the random collapse.

• Anchor choice changes the measured semantics. Coherent delta heads number 41/96 on Charts, 74/96 on Flowers102, 71/96 on EuroSAT, 61/96 on Aircraft, and 16/72 on ViT-B/16. Replacing financial text anchors with generic ones while holding images and hyperparameters fixed drops Charts from 41 coherent heads to zero, even though the null tune converges from loss 0.527 to 0.037.

• Pretrained readouts are ambiguous. A naive pretrained audit finds three coherent heads including color and anatomy matches for financial phrases; adding 21 explicit competitors and 41 chart-anchored probes reduces selected vectors from 59 to 37 and vectors passing both criteria from four to one, with the survivor still ambiguous between a portrait and the financial sense.

• A pre-registered hypothesis failed. The original audit predicted VO recovery at most 0.25 and MLP recovery at least 0.50; all five VO recoveries exceed 0.25, so the paper reports the observed redundancy instead.

Methodology in Plain English

The authors take a pretrained CLIP model and fine-tune it on five image classification tasks: synthetic financial charts, Flowers102, EuroSAT (satellite land use), and FGVCAircraft, using ViT-L/14 for all four and repeating Flowers102 on ViT-B/16. Only the final six visual blocks are trained; the text side, final visual LayerNorm, projection, and logit scale stay frozen, so class names act as fixed "text anchors."

They then ask two complementary questions about each group of weights (attention value–output, attention query–key, MLP, and two LayerNorms):

1. Reversion – put those weights back to their pretrained values and see how much performance drops. This measures necessity under that specific edit.
2. Retention – keep only those fine-tuning updates and revert everything else, then see how much of the original improvement returns. Because the pretrained network is still there, this measures how much behavior the retained update group can recover on its own.

To check that any recovery isn't just an artifact of keeping some weights, they build control masks that keep exactly as many scalar parameters per layer (2w², matching the VO count) but chosen at random, over ten fixed seeds. They also look at which pretrained-model errors each retained model fixes, to see whether VO and MLP recover the same behavior.

For the semantic part, they decompose each attention head's effective VO weight change with SVD, map the leading directions into CLIP's joint embedding space, and greedily match them against a dictionary of 1,076 generic text atoms, 21 confounder atoms, and 41 domain probes, with a statistical threshold calibrated on 5,000 random directions. Directions that pass are called "coherent." They then delete those directions from the output projection and compare the resulting collapse in recovery against same-head, same-count random deletions.

Why This Matters

Impact on research: The paper shows that ablation results alone can mislead. A near-zero accuracy drop after removing a weight group does not establish that the group carries no useful adaptation, because the rest of the network can support the same predictions. This directly challenges a common inference pattern in interpretability and model-editing work, and it offers a concrete control protocol (pair every removal with a retention, add matched-random subsets, and report continuous metrics).

Real-world applications:

• Model auditing and capability removal: Determining whether a fine-tuned capability truly depends on particular weights matters for safety evaluations that try to strip unwanted behaviors.
• Model editing and unlearning: Practitioners deleting or patching weights need to know whether the edit removes the behavior or just lets redundant pathways compensate.
• Model merging and delta compression: Knowing which delta subsets recover most of the improvement informs which weights to keep when combining or compressing fine-tuned models.
• Regulated domains like finance: The synthetic chart task is a controlled stand-in for specialized visual concepts, relevant to auditing models trained on domain-specific imagery.

Industry relevance: Teams that ship fine-tuned vision models can use the reversion/retention pairing as a cheap diagnostic before claiming a weight edit removed a capability, and the finding that semantically aligned directions are not uniformly the most causally important is a caution for interpretability-driven product and compliance claims.

Future Directions

• Vary the adaptation recipe. The evidence covers one recipe (final six visual blocks, frozen text anchors). Learning-rate changes, broader trainable surfaces, per-layer retention, and other model families would test how general the redundancy finding is.
• Try low-rank adaptation such as LoRA. Restricting adaptation to particular subspaces changes which update groups exist; the authors explicitly flag this as untested.
• Improve direction-level interventions. Current readout-guided removal acts through output-projection surgery that also removes pretrained mass and is sequential rather than direction-only; a cleaner direction-only retention operator remains open.
• Resolve semantic ambiguity in pretrained models. The single surviving pretrained chart hit is ambiguous between a literal portrait and the financial pattern, and the LAION-2B OpenCLIP checkpoint's four of 96 hits could not be traced in detail, leaving that interpretation unresolved.

Target Audience

Researchers and graduate students in mechanistic interpretability, weight-space analysis, and model editing; practitioners who audit or prune fine-tuned vision–language models; and safety or compliance engineers who need to justify claims that a capability has been removed from a model. Readers without background in CLIP internals or transformer attention decomposition will find the methodology sections demanding, though the core reversion-versus-retention argument is accessible.

Authors’ abstract

Fine-tuning adapts pretrained models to new tasks, but identifying which weight changes support the resulting behavior remains difficult. Weight-space interpretability assigns semantic descriptions to these changes, called fine-tuning deltas. Such descriptions leave open how much task performance depends on the identified updates, particularly when other updates can support the same predictions. We address this gap by pairing reversion, which restores selected parameters to their pretrained values, with retention, which keeps only selected fine-tuning updates. We evaluate Contrastive Language-Image Pretraining (CLIP) models on synthetic financial charts, Flowers102, EuroSAT, and FGVCAircraft with ViT-L/14, and on Flowers102 with ViT-B/16, using a shared recipe that tunes the final six visual blocks. Our contributions are threefold. First, attention value-output (VO) updates alone recover 43-87% of the accuracy improvement, although reverting them leaves chart accuracy unchanged. Cross-entropy loss preserves this contrast on Charts and EuroSAT, while correct-class score margins reveal additional contributions. Second, VO weights recover more loss improvement than all ten equally sized, layer-matched random subsets in each setting. VO and multilayer perceptron (MLP) retentions jointly correct 25-81% of pretrained errors, demonstrating overlapping recovered behavior, which we call operational redundancy. Third, removing semantically selected directions reduces VO-retained recovery more than matched-random removal in all five settings. Together, these results distinguish semantic alignment, removal cost, and retained recovery as complementary evidence about the functional role of fine-tuning updates within the evaluated CLIP recipe.