First is Not Really Better Than Last: Evaluating Layer Choice and Aggregation Strategies in Language Model Data Influence Estimation

AI summary

Overview

Research area: Interpretability and data attribution for Large Language Models, specifically gradient-based influence functions and how the choice of model layers affects them.

Technical level: Advanced. The paper combines formal definitions of influence functions, a theorem with proof, Hessian-based approximations, and large-scale empirical benchmarking.

Scope: A theoretical and empirical re-examination of which layers of a language model should be used to compute training-data influence scores, how those per-layer scores should be combined, and how to evaluate influence methods without retraining.

What This Paper Is About

Computing influence scores across every parameter of a modern LLM is computationally infeasible, so practitioners restrict influence computations to a subset of layers. A widely cited earlier study by Yeh et al. (2022) concluded that the first (embedding) layers are the most informative, justified by a "cancellation effect" hypothesis. This paper argues that the cancellation effect is an unreliable guide, that middle attention layers are better estimators of influence, and that how influence scores are aggregated across layers matters as much as which layers are chosen.

Key Contributions

1. A refutation of the cancellation effect. The authors provide both theoretical and empirical evidence that the cancellation metric used by Yeh et al. (2022) is an unreliable indicator of layer usefulness. They prove a theorem ("Cancellation Can Improve Influence Estimation") showing that including high-cancellation weights can strictly increase the separation between noisy and clean samples — the opposite of what the cancellation hypothesis predicts.

2. A large-scale empirical map of layer effectiveness. Using noisy versions of eight GLUE datasets and four models — RoBERTa-Large, LLaMA-3.2 1B, Qwen-2.5 1.5B, and Mistral 7B — they quantify downstream accuracy after filtering out the 30% least influential samples using different layer groups.

3. New aggregation strategies. They introduce Rank and positional Vote aggregation as alternatives to the standard averaging of influence scores across layers and validation samples, and show these frequently outperform averaging.

4. New evaluation proxies. They propose the Noise Detection Rate (NDR) and Area Under the Curve (AUC) as cheap, retraining-free indicators of how well an influence method will perform after filtering, and measure their correlation with actual post-filtering task accuracy.

Main Findings

• The classification head is the weakest choice. Across models and influence methods, filtering with scores from the classification head (CL) improves downstream accuracy less than other layers, and CL is shown to be highly susceptible to influential noise — the one point on which the authors agree with Yeh et al. (2022).

• Embeddings are not universally best. TracIn on RoBERTa-Large performs best when the embedding layer (WE) is used, but for larger models the first or second attention layer groups are generally better, both in performance and computational cost.

• Middle/early attention layers win in larger models. For Mistral 7B, DataInf achieves its strongest results on layers 08–15 (0.71 win rate), 00–07 (0.69), and 16–23 (0.51). For Qwen-2.5 1.5B, the strongest results come from Cosine on layers 07–13 and DataInf on layers 00–06 and 07–13, all ranked top-1, with middle layers (07–13) consistently outperforming other layers.

• RoBERTa-Large behaves differently. For this smaller model, filtering performance peaks in later attention layers: DataInf on layers 18–23 reaches a 0.70 win rate, with Cosine on layers 12–17 (0.56) and TracIn on the embedding layer (0.51) as the closest competitors. The two embedding-based variants TracIn_we and TracIn¹⁰_we score nearly identically (0.47).

• Llama-3.2 1B is the hardest case. No influence configuration surpasses the uniform-removal baseline (Random). The best outcomes are DataInf on layers 04–07 (0.64 win rate) and 00–03 (0.58), comparable to Random (0.64). Nevertheless, noise distribution analysis shows mislabeled samples concentrate at both extremes of the influence range.

• Best layers beat worst layers by 10–15%. For Mistral 7B after 30% filtering (averaged over 10 runs), the gap between the best and worst layer configurations is 10–15%, with a Friedman p-value of 1e-142, indicating statistically significant differences.

• Cancellation correlates weakly or not at all with performance. Reported Spearman correlations between the cancellation metric C and post-filtering accuracy range from -0.3 to 0.3 across configurations, with many values marked non-significant. Layer-group cancellation also masks individual-parameter cancellation: WE groups have some of the smallest mean values (2.2 ± 0.3 for RoBERTa-Large, 2.9 ± 0.3 for Llama-3.2 1B, 3.5 ± 0.3 for Mistral 7B) yet contain individual weights with infinite cancellation scores.

• Layer scores cluster into three groups. Correlations across layers reveal early, middle, and late groups; within a group layers assign similar scores, while cross-group correlations are weak — indicating layers often compensate for one another.

• Voting improves aggregation. Vote consistently improves accuracy for DataInf and TracIn across most datasets and layers while degrading it for Cosine. It raises the win rate of DataInf on layers 00–07 to 0.84, lifts the rank of TracIn CL from 10 (Mean) to 2 (Vote) on RoBERTa, and from 12 (Mean) to 1 (Vote) on Mistral.

• The number of votes matters. For Qwen-2.5 1.5B with the LoRA Value B module, DataInf and TracIn are maximized for k in [10, 50], while Cosine reaches a local minimum around k ≈ 50 and its maximum at larger k.

• NDR is a better proxy than AUC or cancellation. NDR at 30% generally shows higher correlation with final task accuracy than AUC. Correlations are moderate-to-strong overall but fall in the 0.5–0.7 range in many configurations, which the authors state is not consistently reliable. Vote aggregation achieves stronger correlations of 0.8–0.9, suggesting that appropriate aggregation improves proxy predictiveness.

Methodology in Plain English

The authors follow and extend a five-stage evaluation pipeline used in prior influence-function work:

1. Inject synthetic noise into training data and initialize the model from compressed embeddings.
2. Fine-tune on the noisy data, selecting checkpoints with the lowest validation loss (which the authors verify outperforms final-epoch or accuracy-based selection).
3. Compute influence values across all tunable layers, using three influence functions: TracIn, Cosine, and DataInf.
4. Partition the model into the embedding layer (WE), four attention groups, and the classification head (CL), aggregating influence per training sample.
5. Remove the 30% least influential samples and retrain, measuring effectiveness by the best test accuracy.

Each configuration is repeated over 10 seeds. Configurations are compared pairwise across datasets, checkpoints, and seeds: configuration A beats B if it performs better under identical settings, and win rates are aggregated into a score matrix from which Pareto fronts identify which methods dominate. Experiments span eight GLUE datasets — QNLI, MRPC, QQP, SST-2, CoLA, MNLI, RTE, and STS-B — covering sentence similarity, paraphrase detection, sentiment analysis, linguistic acceptability, and natural language inference. Beyond the standard averaging aggregation, they test ranking-based aggregation (which sums per-layer, per-validation-sample ranks and ignores incorrectly predicted validation samples) and positional voting (where each validation sample and layer assigns k votes to the least influential training datum, k−1 to the next, and so on). They then test whether the Noise Detection Rate and AUC — both computed purely from the distribution of known mislabeled samples across the influence ranking — predict the accuracy actually obtained after retraining.

Why This Matters

Impact on research. The paper challenges a specific, influential claim about layer selection in LLM influence estimation and supplies both a counterexample theorem and broad empirical evidence. It also reframes aggregation as a first-class design choice rather than a fixed default, and offers cheap proxy metrics that could reduce reliance on expensive retraining loops in future influence-method development.

Real-world applications:

• Training data auditing and curation — identifying mislabeled, anomalous, or out-of-distribution samples in large datasets before training.
• Detecting harmful or malicious data — influence functions have been explored for harmful data detection and backdoor identification, areas where prior work (Li et al., 2025) found default implementations performed poorly.
• Lowering the cost of dataset filtering — choosing early-to-middle attention layers instead of full-model gradients makes influence estimation more tractable for billion-parameter models.
• Interpretability and trust/safety — linking model behavior back to specific training samples supports explanations of model decisions.

Industry relevance. Teams that fine-tune LLMs, especially with LoRA, face the same cost trade-off this paper studies. The finding that layer choice and aggregation strategy can move filtering accuracy by 10–15% — and that a simple voting scheme can raise win rates to 0.84 — translates directly into cheaper, more effective data-cleaning pipelines. The NDR metric allows these teams to screen influence methods without repeatedly retraining models.

Future Directions

• Better aggregation operators. The authors explicitly suggest that future work could explore efficient unsupervised or multi-objective aggregation methods to exploit influence signals and further improve filtering performance.
• Closing the gap for weaker models. Llama-3.2 1B is the case where no influence configuration beat random removal. Understanding why, and whether other layer or aggregation choices help, remains open.
• Why middle layers work. The paper notes a convergence with knowledge-editing studies (ROME, R-ROME, MEMIT, EMMET) that also find the strongest discriminative signals in middle layers, but does not fully explain the mechanism.
• More reliably predictive proxies. NDR and AUC correlate with downstream accuracy only in the 0.5–0.7 range for standard averaging, so a consistently reliable retraining-free metric is still missing.

Target Audience

Researchers and graduate students working on LLM interpretability, data attribution, and influence functions; practitioners who curate and filter training data for large language models; and engineers fine-tuning models with LoRA who need to decide which layers to instrument and how to combine per-layer signals. Readers should be comfortable with gradient-based influence formulations and Hessian approximations, since the paper includes a formal theorem and layer-wise gradient analysis.

Authors’ abstract

Identifying how training samples influence/impact Large Language Model (LLM) decision-making is essential for effectively interpreting model decisions and auditing large-scale datasets. Current training sample influence estimation methods (also known as influence functions) undertake this goal by utilizing information flow through the model via its first-order and higher-order gradient terms. However, owing to the large model sizes of today consisting of billions of parameters, these influence computations are often restricted to some subset of model layers to ensure computational feasibility. Prior seminal work by Yeh et al. (2022) in assessing which layers are best suited for computing language data influence concluded that the first (embedding) layers are the most informative for this purpose, using a hypothesis based on influence scores canceling out (i.e., the cancellation effect). In this work, we propose theoretical and empirical evidence demonstrating how the cancellation effect is unreliable, and that middle attention layers are better estimators for influence. Furthermore, we address the broader challenge of aggregating influence scores across layers, and showcase how alternatives to standard averaging (such as ranking and vote-based methods) can lead to significantly improved performance. Finally, we propose better methods for evaluating influence score efficacy in LLMs without undertaking model retraining, and propose a new metric known as the Noise Detection Rate (NDR) that exhibits strong predictive capability compared to the cancellation effect. Through extensive experiments across LLMs of varying types and scales, we concretely determine that the first (layers) are not necessarily better than the last (layers) for LLM influence estimation, contrasting with prior knowledge in the field.