Order-Level Attention Similarity Across Language Models: A Latent Commonality

AI summary

Overview

Research area: Natural Language Processing — interpretability and knowledge transfer for transformer-based language models (LMs).

Technical level: Advanced. The paper assumes familiarity with attention mechanisms, Attention Rollout, residual connections, adapters/LoRA-style tuning, and the distinction between causal language models (CLMs) and masked language models (MLMs).

Scope: The paper proposes Order-Level Attention (OLA), an order-wise decomposition of Attention Rollout that makes attention weights comparable across models, shows that OLA at the same order is similar across LMs and encodes syntactic knowledge, and uses this to build a training-free cross-model adapter transfer method.

Code is available at https://github.com/jinglin-liang/OLAS.

What This Paper Is About

Different transformer LMs have different layer counts, head counts, architectures, and training data, so their attention weights are not directly comparable — each model's attention has its own meaning. The authors ask whether the context aggregation patterns of different pretrained LMs nonetheless share a common structure. They answer by re-expressing attention in a form whose semantics are the same across models, then show this shared structure exists and can be exploited to move an adapter from one LM to another without any retraining.

Key Contributions

1. OLA (Order-Level Attention). A representation derived from decomposing Attention Rollout into paths grouped by how many times information passes through an attention module. Because OLA is defined by aggregation count rather than by layer index or head index, it unifies attention across models with different depths and head counts into comparable, equivalently-defined matrices (§3.1).

2. Two empirical findings about OLA. (a) OLA of the same order is significantly similar across different LMs, a phenomenon the authors name Order-Level Attention Similarity (OLAS), supported by qualitative visualizations (§3.2) and two quantitative evaluation designs (§3.3) over 12 LMs. (b) OLA implicitly encodes syntactic knowledge, demonstrated by predicting syntactic dependencies directly from OLA with an auxiliary model (§3.4).

3. Transferable OLA Adapter (TOA). A training-free cross-LM adapter transfer method that treats OLA as a unified syntactic feature representation, trains an adapter on OLA from a source LM, and applies it unchanged to unseen target LMs (§4.1). The authors state they are the first to achieve training-free cross-model adapter transfer.

4. Extensive transfer evaluation. TOA is evaluated on four tasks — relation extraction (RE), named entity recognition (NER), dependency parsing (DP), and part-of-speech tagging (POS) — across CLMs and MLMs, compared against the target models' zero-shot performance (§4.2).

Main Findings

• Same-order OLA is similar across models. Visualizations of first- and second-order OLA from Qwen2-1.5b and Llama3.2-3b on the same text are highly similar, while OLA from different texts is clearly distinguishable. Attention Rollout, by contrast, shows a consistent pattern across different texts.

• Attention sinks are weakest in low-order OLA. Attention Rollout (the weighted sum of all orders) shows the most severe attention sinks; first-order OLA shows the least sinking, and second-order shows more than first-order. The authors interpret higher-order OLA as potentially ineffective components carrying similar biases.

• Visual-classifier test. A ResNet-18 classifier trained on source LMs' OLA maps (same text mapped to the same class) was tested on target LMs' OLA maps. First-, second-, and third-order OLA all achieved high accuracy across MLMs and CLMs, with first-order OLA exceeding 90% in multiple settings. For example, with the source set {Llama3.2-3b, Llama3.1-8b, Qwen2-1.5b, Qwen2-7b}, first-order OLA reached 94.60% on Llama3.2-3b and 94.10% on Llama3.1-8b; on MLMs, first-order reached 95.20% (Electra-base) and 95.80% (Electra-large) with the source set {Bert-base, Bert-large, Roberta-base, Roberta-large}. Baselines (Attention Rollout, IRNL, ALTI) also showed cross-model similarity but less prominently than OLA.

• Retrieval test. Using SSIM between target-LM OLA queries and source-LM OLA candidates, first-order CLM results showed the lowest Hits@5 above 89% and the highest above 97%; MLM performance was weaker than CLM but still substantial. For example, G-2b→L-3b reached 95.30 / 97.10 and L-3b→Q-1b5 reached 92.90 / 96.10. Second- and third-order retrieval results are reported in the paper's Table 12.

• OLA carries syntactic knowledge. An auxiliary dependency parsing network trained on OLA achieved first-order UAS/LAS of 63.58/48.24 (Qwen2-1.5b), 62.25/45.95 (Gemma2-2b), 62.98/48.19 (Llama3.2-3b) for CLMs, and 81.29/72.16 (Bert-base), 80.00/70.44 (Roberta-base), 81.23/72.63 (Electra-base) for MLMs. Performance consistently decreased from first- to second- to third-order, and Attention Rollout scored lowest (e.g., 53.77/35.57 for Llama3.2-3b; 50.35/34.02 for Electra-base).

• TOA beats zero-shot in most settings. For MLMs, TOA surpassed zero-shot on all four tasks for all source–target pairs. For CLMs, TOA exceeded zero-shot for all source–target pairs on RE and DP but underperformed on a small fraction of NER and POS cases (less than 6% of total scenarios), mainly involving larger, high-capacity target LMs such as Gemma2-9B.

• Concrete transfer example. Transferring TOA trained on LLaMA3-3B (Llama3.2-3b) to Qwen2-1.5B raised relation prediction accuracy from 7.69% (zero-shot) to 34.90%, versus Qwen2-1.5B's own self-transfer score of 34.90%.

• Cross-model degradation is small. Transferring TOA from BERT-base to BERT-large reached 29.90% accuracy on RE, which is 93% of BERT-large's self-transfer performance (32.29%).

• TOA works better on MLMs and on syntax-heavy tasks. MLMs outperformed CLMs, which the authors attribute to bidirectional attention capturing richer context than unidirectional attention. TOA performed better on DP and POS than on the semantics-driven RE and NER, suggesting OLA primarily encodes syntactic structure rather than semantics.

• OLAS is tied to pretrained parameters. Controlled experiments varying dataset and preprocessing showed OLAS persists; perturbing model parameters made the phenomenon disappear, indicating it is not an artifact of data or experimental setup.

Methodology in Plain English

Each transformer layer mixes information in two ways: through the attention module, and through a residual connection that skips it. Unrolling this, an N-layer model contains 2^N different paths by which information can flow from one token to another, and the standard Attention Rollout multiplies all layer matrices together, effectively summing everything.

The problem with that sum is that it is dominated by attention sinks — nearly all the mass concentrates on a few unimportant tokens such as <bos>, and the resulting map looks nearly identical regardless of which text is fed in. The authors' fix is to split the Rollout sum by how many attention modules a path passes through. The 0th-order term is the identity matrix (all residual connections). The first-order term averages paths with exactly one attention step, the k-th order term covers paths with k attention steps, and the full Rollout is a weighted sum of these orders with binomial coefficients.

This decomposition has a key property: "first-order" means the same thing in every model, regardless of how many layers or heads it has. That makes attention comparable across models.

To test the similarity claim, the authors run two evaluations on 12 LMs (six CLMs: Qwen2-1.5B, Qwen2-7B, Gemma2-2B, Gemma2-9B, Llama3.2-3B, Llama3.1-8B; six MLMs: Bert-base-cased, Bert-large-cased, Roberta-base, Roberta-large, Electra-base, Electra-large). The first trains a ResNet-18 image classifier where all OLA maps for the same text — from any source LM — share a label, then tests whether it correctly classifies OLA from held-out target LMs. The second casts similarity as image retrieval: target-LM OLA maps are queries, source-LM OLA maps are the gallery, SSIM scores rank candidates, and Hits@1 / Hits@5 measure how often the same-text match is found.

To test whether OLA means anything linguistically, they train a small dependency parser whose only input is OLA and whose target is the gold syntactic dependency annotation. If parsing works from OLA alone, OLA must encode syntax.

Finally, TOA trains an adapter on OLA (the experiments stack first- and second-order OLA as input features) for a downstream task on a frozen source LM, then applies that adapter with no changes to a target LM, comparing against the target LM's zero-shot prompted performance.

Why This Matters

Research impact. Prior work on attention analysis focuses on a single model or head, which emphasizes each model's idiosyncrasies. This paper reframes the question around what models share, and shows that the shared part is not merely qualitative — it is strong enough to be measured, and strong enough to carry a trained module across model boundaries. It also hints that OLA similarity could serve as an indicator for measuring how similar two LMs are, since the authors find similarity varies by source–target combination and is influenced by factors such as data and architecture.

Real-world applications.

• Cheap adaptation of large models. Train an adapter on a small model, then transfer it to a larger one, avoiding the cost of tuning the adapter on the larger model.
• Open-to-closed model transfer. Train an adapter on an open-source model and transfer it to a closed-source model, so the closed model gains knowledge from the data while model and data remain isolated — a privacy-preserving setup the authors explicitly motivate.
• Lower adaptation cost. Because TOA is training-free at transfer time, no training data and no parameter updates are needed on the target side.
• Indexing or matching across model families. Since OLA distinguishes texts while being stable across models, it could serve as a model-agnostic feature representation for sentences.

Industry relevance. The economics of fine-tuning are the recurring obstacle in deploying LMs: results produced on one model cannot be reused on another, forcing repeated adaptation. A method that lets an already-trained adapter work on a new model without retraining addresses that cost directly, and the paper's emphasis on syntax-heavy tasks (DP, POS) versus semantics-driven tasks (RE, NER) gives practitioners a concrete signal about where to expect gains.

Future Directions

• Beyond first- and second-order OLA. The experiments use stacked first- and second-order OLA as adapter input, and the authors note that other configurations — first-order only, or other order combinations — can be chosen per task. Which order combinations are optimal for which tasks is left open.

• Explaining the failures on large high-capacity targets. For CLMs, TOA underperformed zero-shot on a small fraction of NER and POS cases (less than 6% of total scenarios), predominantly with large target models such as Gemma2-9B. What causes this, and whether it can be corrected, is unresolved.

• Closing the syntax–semantics gap. TOA performs better on syntax-dependent tasks than on semantics-driven ones, which the authors read as evidence that OLA primarily encodes syntactic structure. Whether a semantics-oriented counterpart representation exists is an open question.

• OLA similarity as a model-comparison metric. The authors observe that OLA similarity differs across source–target pairs and is influenced by training factors such as data and architecture, and propose OLA as a potential indicator for evaluating LM similarity. How such a metric would behave over a wider range of models and languages is not reported.

Target Audience

Researchers and engineers working on transformer interpretability, attention analysis, and parameter-efficient fine-tuning or adapter transfer. The paper is most useful to readers already comfortable with attention mechanics, Attention Rollout, and adapter methods, since the derivation and the evaluation designs build directly on those concepts. Practitioners facing repeated adaptation costs across multiple LMs — particularly those working with both CLMs and MLMs or mixing open- and closed-source models — will find the TOA results and the task-dependent findings most immediately applicable.

Authors’ abstract

In this paper, we explore an important yet previously neglected question: Do context aggregation patterns across Language Models (LMs) share commonalities? While some works have investigated context aggregation or attention weights in LMs, they typically focus on individual models or attention heads, lacking a systematic analysis across multiple LMs to explore their commonalities. In contrast, we focus on the commonalities among LMs, which can deepen our understanding of LMs and even facilitate cross-model knowledge transfer. In this work, we introduce the Order-Level Attention (OLA) derived from the order-wise decomposition of Attention Rollout and reveal that the OLA at the same order across LMs exhibits significant similarities. Furthermore, we discover an implicit mapping between OLA and syntactic knowledge. Based on these two findings, we propose the Transferable OLA Adapter (TOA), a training-free cross-LM adapter transfer method. Specifically, we treat the OLA as a unified syntactic feature representation and train an adapter that takes OLA as input. Due to the similarities in OLA across LMs, the adapter generalizes to unseen LMs without requiring any parameter updates. Extensive experiments demonstrate that TOA's cross-LM generalization effectively enhances the performance of unseen LMs. Code is available at https://github.com/jinglin-liang/OLAS.