Abstraction Induces the Brain Alignment of Language and Speech Models

AI summary

Overview

Research area: NeuroAI — specifically, the alignment between representations learned by large language models and speech/audio models and measured human brain responses to natural language.

Technical level: Intermediate. The paper assumes familiarity with encoding models, linear probing, Transformers, and neural response measurement (fMRI, ECoG), but the core argument is conceptual rather than mathematically dense.

Scope: A single-sentence scope: the paper argues, using six language models and three speech models across fMRI and ECoG data, that the reason intermediate model layers best predict brain activity is that those layers hold the richest meaning abstractions of the input, as tracked by a peak in the representational intrinsic dimension (denoted $I_d$) — not because of next-word prediction.

What This Paper Is About

Language and speech models are known to predict brain activity when the intermediate hidden layers — not the output layers — are linearly mapped onto neural responses. It has been unclear why the middle layers are optimal. This paper tests the hypothesis that the driver is the richness of linguistic abstraction built in those middle layers, and that next-token prediction (surprisal) does not explain the pattern. The authors link a layer's brain-predictivity to its intrinsic dimension, a measure of how many nonlinear degrees of freedom underlie its representations.

Key Contributions

1. Demonstrating that next-token predictivity is not the driver. The authors show that surprisal decreases monotonically across layers (the last layer always has the lowest surprisal), while the best brain-predictive layer falls between 40–70% of processing, and that surprisal correlates weakly or inconsistently with layerwise encoding performance across model families.
2. Establishing intrinsic dimension ($I_d$) as a strong predictor of layerwise encoding performance, in both language and speech models, for both fMRI and ECoG, with the $I_d$ peak generally coinciding with the encoding-performance peak (usually 0–1 layers apart; worst case 5 layers out of 40 in OPT-13b).
3. Extending the $I_d$-peak-as-abstraction-phase finding from LLMs to speech models, showing via SentEval and speech-feature probing that higher-order semantic decodability peaks near the $I_d$ peak while surface and acoustic feature decodability declines.
4. Providing causal and control evidence: brain-tuning WavLM increases both its semantic content and its $I_d$, while random Fourier feature spaces constructed to have increasing $I_d$ plateau at much lower encoding performance — showing that high $I_d$ is necessary but not sufficient and is a symptom of learning rich abstractions rather than a mechanism in itself.

Main Findings

• Surprisal does not track the best layer or layerwise performance. Across all models, the global Spearman correlation between layerwise surprisal and encoding performance was $\rho = -0.53$ for fMRI and $0.21$ for ECoG. Within-model correlations varied widely: negative for OPT and Whisper ($p < 0.05$), often non-significant for Pythia, and sometimes positive for WavLM.
• $I_d$ tracks encoding performance within models. In Table 1, layerwise $I_d$ correlations with encoding performance reached $0.90$ (OPT-125m, fMRI), $0.85$ (OPT-13b, fMRI), $0.94$ (Pythia-6.9b, fMRI), $0.97$ (Whisper-large, fMRI), and up to $0.97$ (OPT-125m and OPT-1.3b, ECoG). The one clear exception was WavLM-base-plus, whose encoding performance the authors attribute to low-level similarity to auditory cortex.
• Globally, $I_d$ correlates with encoding performance across models. The text and Table 1 report $\rho = 0.72$ for fMRI and $\rho = 0.43$ for ECoG (the Figure 2 caption states $\rho = 0.76$ for fMRI and $0.43$ for ECoG). Both are reported as significant at $\alpha = 0.05$.
• Mixed-effects modeling favors $I_d$ over surprisal. On fMRI, $I_d$ had an effect size $\beta_{I_d} = 0.63$ while surprisal's effect was explained away ($p$-value $= 0.99$, $\beta_{\text{surprisal}} \approx 0$). On ECoG, both were significant at $\alpha = 0.01$, but $\beta_{I_d} = 0.97$ was roughly four times larger than $\beta_{\text{surprisal}} = 0.26$.
• The $I_d$ peak coincides with semantic, not surface, feature content. Probing showed higher-order semantic decodability peaking in deep layers near the $I_d$ peak, while superficial text features or acoustic features declined across layers.
• The relationship emerges during pre-training. For Pythia-6.9b, a layer-13 $I_d$ peak develops over checkpoints (1K, 2K, 4K, 8K, 16K, 32K, 64K, 143K). fMRI encoding performance peaks at layer 13 and ECoG at layer 12. Across training, $I_d$ and encoding performance correlate at $\rho = 0.96$ (fMRI) and $\rho = 0.64$ (ECoG), both with $p < 1\text{e-}3$.
• Well-predicted voxels and electrodes drive the effect. For OPT-1.3b, the correlation between layerwise encoding performance and $I_d$ increases with how well a unit is predicted: $\rho = 0.58$ for fMRI (Subject UTS03, 95,556 voxels considered) and $\rho = 0.56$ for ECoG (1,268 electrodes across subjects). Restricted to well-predicted units ($r \geq 0.2$ for fMRI; $r \geq 0.1$, roughly the top 25% of electrodes, for ECoG), correlations were $\rho = 0.73$ and $\rho = 0.63$, with the highest values in fronto-temporal language areas.
• Brain-tuning causally increases $I_d$ and semantic content. Finetuning WavLM-base-plus layer 9 on voxelwise fMRI responses raised encoding performance, semantic content, and representational $I_d$ simultaneously.
• High $I_d$ alone is not sufficient. Random Fourier feature maps of increasing extrinsic dimension (128, 256, 512, 1024, 2048) showed a perfect $\rho = 1$ relationship between RFF-$I_d$ and encoding performance, but with severe diminishing returns beyond $I_d$ of 150 and a plateau at $R \approx 0.04$, while LLM or speech layers with lower $I_d$ achieved $R > 0.1$.

Methodology in Plain English

The authors measure four things and compare them across the layers of each model.

First, encoding performance: they extract a representation for each word (or 16-second audio chunk, with 100 ms stride) from every layer, downsample it with a Lanczos filter, add delays of 1, 2, 3, and 4 TRs to account for the slow BOLD hemodynamic response, and train a ridge regression to predict brain activity. Performance is the Pearson correlation $R$ on held-out validation data. For ECoG, they train separate linear models for 128 evenly spaced lags between -2 and +2 seconds and take the best-predicted lag.

Second, intrinsic dimension ($I_d$): they estimate this with GRIDE on $N = 10000$ random 20-word contexts from The Pile for LLMs and $N = 10000$ random audio chunks of at most 20 seconds from LibriSpeech for speech models, repeated over 5 bootstraps and averaged. Crucially, this is measured on generic in-distribution data, so it describes the layer's general function rather than its response to any particular stimulus. They note that linear dimensionality estimators gave weaker results.

Third, linguistic content: layerwise linear probes using SentEval for LLMs and acoustic and semantic feature probes for speech models.

Fourth, surprisal: they fit each intermediate layer to the vocabulary with an affine map and compute next-token prediction error, as a competing explanation.

Finally, they run two manipulations: brain-tuning WavLM on fMRI responses to test causality, and random Fourier features to test whether $I_d$ alone can produce brain predictivity.

Why This Matters

Impact on research. The paper reframes a widely observed but unexplained phenomenon in NeuroAI. It argues against the "predictive coding" explanation of brain-model alignment and in favor of abstraction as the driver, and it introduces $I_d$ as a practical diagnostic for when and where in a network rich linguistic features are constructed. It also supports a two-phase view of LLM processing: composition and abstraction first, prediction second.

Real-world applications.

• Guiding which layer of a language or speech model to extract features from for any downstream use involving human neural or behavioral data.
• Improving brain-computer interfaces that decode speech or language from neural signals, by selecting representations with the richest semantic content.
• Informing model compression and layer-selection strategies for clinical or assistive neurotechnology where compute is limited.
• Providing a diagnostic for evaluating whether a model's representations capture meaning versus surface form.

Industry relevance. Anyone selecting intermediate representations from foundation models for transfer — in speech recognition, neurotechnology, cognitive modeling, or representation-based evaluation — has a concrete signal ($I_d$) for layer choice. The finding that autoregressive objectives leave out readily learnable nonlinear features useful for predicting brain activity also motivates non-autoregressive or auxiliary training objectives.

Future Directions

• Identify which specific features $I_d$ encodes. The authors state that $I_d$ is a coarse, dataset-level measure indicating how many degrees of freedom exist but not what they are, and call an explicit, interpretable account an open problem.
• Complete the causal picture. The paper notes that the relationship between intrinsic dimensionality and encoding performance is "rife with co-dependencies that are difficult to disentangle," and that brain-tuning provides only one element of causal analysis.
• Develop better methods for extracting task-relevant structure from naturalistic data, which the authors argue is key to pushing encoding models beyond current limits.
• Test whether language modeling is the only task that induces this abstraction. The authors describe language modeling as a task "complex enough (but perhaps not the only)" to require it.

Target Audience

Researchers in NeuroAI and computational neuroscience studying encoding models of language and speech; NLP researchers interested in how representations evolve across layers and during pre-training; and machine learning practitioners who need principled criteria for selecting intermediate model representations for transfer to neural or behavioral prediction tasks. Readers without background in fMRI or ECoG methodology will need to consult the cited dataset papers for context.

Authors’ abstract

Research has repeatedly demonstrated that intermediate hidden states extracted from large language models and speech audio models predict measured brain response to natural language stimuli. Yet, very little is known about the representation properties that enable this high prediction performance. Why is it the intermediate layers, and not the output layers, that are most effective for this unique and highly general transfer task? We give evidence that the correspondence between speech and language models and the brain derives from shared meaning abstraction and not their next-word prediction properties. In particular, models construct higher-order linguistic features in their middle layers, cued by a peak in the layerwise intrinsic dimension, a measure of feature complexity. We show that a layer's intrinsic dimension strongly predicts how well it explains fMRI and ECoG signals; that the relation between intrinsic dimension and brain predictivity arises over model pre-training; and finetuning models to better predict the brain causally increases both representations' intrinsic dimension and their semantic content. Results suggest that semantic richness, high intrinsic dimension, and brain predictivity mirror each other, and that the key driver of model-brain similarity is rich meaning abstraction of the inputs, where language modeling is a task complex enough (but perhaps not the only) to require it.