Backbone-Informed Adaptation after EEG Pretraining: A Capacity-Controlled Study

AI summary

Overview

Research area: Machine learning for electroencephalography (EEG) — specifically, how to adapt pretrained EEG foundation models to downstream decoding tasks.

Technical level: Intermediate. Readers should be comfortable with the idea of a frozen pretrained encoder, adapters, attention, and token-wise multilayer perceptrons; no deep EEG signal-processing background is required.

Scope: A capacity-controlled empirical comparison of a "backbone-informed" (native-axis attention) adapter against a parameter-matched token-wise MLP adapter, evaluated on two EEG foundation models (CBraMod and LaBraM) across three task families.

What This Paper Is About

EEG foundation models are pretrained to mix information across electrode channels and short time segments (temporal patches), so a natural instinct is to design downstream adapters that follow that same channel-and-patch structure. This paper asks whether such a backbone-informed adapter actually beats a much simpler adapter that transforms each feature vector independently — once you control for how many trainable parameters each one has. The authors keep the pretrained encoder frozen and compare the two designs head-to-head, plus against a frozen-backbone probe and during full fine-tuning, to see whether the structured design earns its keep.

Key Contributions

1. A capacity-matched head-to-head protocol. The authors approximately match trainable adapter parameter counts (within a 5% relative mismatch tolerance) between a native-axis attention adapter and a token-wise MLP control, holding the checkpoint, data split, seed, classifier head, and training recipe fixed within each pair. Actual counts are 59,610 vs. 59,949 for CBraMod (0.569% mismatch) and 85,810 vs. 86,014 for LaBraM (0.238%).

2. Backbone-specific native adapters for two EEG foundation models. They build native adapters that follow CBraMod's split channel/temporal feature halves and LaBraM's channel-by-patch token grid, mixing information along the same axes the encoder uses.

3. A three-question evaluation structure. RQ1 asks whether the native adapter beats the capacity-matched token-wise control; RQ2 asks whether it beats a frozen-backbone probe (no adapter); RQ3 is a secondary analysis of whether it helps when the backbone itself is fully fine-tuned.

4. Diagnostic follow-ups. A branch-preserving MLP control (native branch layout, no attention), a sequence-encoder training sensitivity analysis on sleep staging, and class-level error analysis on EEG event classification to locate where the designs diverge.

Main Findings

• No consistent advantage for the backbone-informed adapter. Across the six primary matched comparisons, the native adapter never shows a stable win over the capacity-matched token-wise control; the preferred design depends on the backbone, the task, and the downstream training protocol.

• CBraMod differences are within seed noise. Mean RQ1 effects range from −0.004 to +0.003 across balanced accuracy and macro-F1, smaller than the corresponding across-seed standard deviations. Example balanced accuracies: FACED probe .3777 ± .0023, native .3910 ± .0039, token-wise .3879 ± .0031 (RQ1 effect +.0030 ± .0045); ISRUC .7562 / .7519 / .7522; TUEV .4656 / .4548 / .4580.

• LaBraM favors the token-wise control on emotion recognition and event classification. On FACED, native balanced accuracy is .1613 ± .0097 vs. .1951 ± .0028 for token-wise (−.0338 ± .0116), and macro-F1 .1406 vs .1940 (−.0533 ± .0070). On TUEV, native is .3916 ± .0182 vs .4307 ± .0010 (−.0391 ± .0174), and macro-F1 shows a −.0653 gap. All three seeds favor token-wise in both settings.

• The one apparent native win is protocol-sensitive. On LaBraM–ISRUC sleep staging (with a newly initialized frozen sequence encoder), the native adapter leads by +.0340 balanced accuracy and +.0345 macro-F1 (.7453 vs .7114 and .7322 vs .6977). When the downstream sequence encoder is trained jointly, that gap shrinks to 0.23 balanced-accuracy points and 0.22 macro-F1 points, and macro-F1 changes sign on one seed. The native adapter still exceeds the branch-preserving MLP by 0.37 BA and 0.30 F1 points on average.

• Improving on the probe does not imply the better adapter. On LaBraM–TUEV, the native adapter beats the frozen probe (+.0258 ± .0182 balanced accuracy, +.0193 ± .0260 macro-F1) yet loses to the token-wise control on both metrics.

• Branch layout without attention performs like the token-wise control. Against the branch-preserving MLP, LaBraM native is weaker by .0384 (FACED BA), .0540 (FACED F1), .0395 (TUEV BA), and .0660 (TUEV F1), and stronger by .0317 (ISRUC BA) and .0319 (ISRUC F1). Branch-preserving MLP minus token-wise differences are small: +.0046, +.0007, +.0023, +.0026, +.0003, +.0007.

• Adding the adapter during full fine-tuning gives no consistent benefit (RQ3). It improves mean balanced accuracy and macro-F1 on CBraMod–TUEV (BA +1.27 ± 3.70 points, F1 +2.49 ± 8.58 points, standard deviations exceeding the means), CBraMod–SEED-V, and LaBraM–PhysioNet-MI, but reduces both metrics on CBraMod–FACED.

• TUEV errors concentrate in two classes. LaBraM's native-adapter deficit is concentrated in generalized periodic epileptiform discharges (GPED) and eye movement (EYEM), where native F1 is lower by 0.114 and 0.190 respectively; CBraMod effects are comparatively small.

• Absolute scores are low for LaBraM on FACED. Balanced accuracy ranges only from .159 to .195 for probe, native, and token-wise, while LoRA and Upper-2 reach .282 and .297 on the same split.

Methodology in Plain English

The authors take two publicly available pretrained EEG models and freeze them. On top of the frozen features they place one of two small trainable modules, then a task classifier:

• Native adapter: splits the feature grid along the encoder's own axes — channels and temporal patches — and runs a small bottleneck attention block along each axis, adding the result back as a residual. For CBraMod this respects its split of features into two halves (channel attention on the first D/2, temporal on the second); for LaBraM the branches use the full token embedding.

• Token-wise control: a plain bottleneck MLP (layer norm, down-projection, GELU, up-projection) applied independently to each channel–patch token, with its width tuned so the parameter count nearly matches the native adapter's. The native bottleneck dimension is fixed at 64, with token-wise widths of 148 (CBraMod) and 213 (LaBraM).

Because the two modules differ in both operator (attention vs. MLP) and token mixing, the comparison is deliberately about complete designs, not isolated components. Every paired run shares the same pretrained checkpoint, split, seed, classifier architecture, optimizer, learning-rate schedule, and residual/dropout/initialization policy, with one training recipe per setting and no per-adapter tuning. Three seeds per condition, checkpoint selection by validation Cohen's κ, and paired per-seed differences (native minus comparison) reported as mean ± standard deviation.

Datasets and design details: FACED (32 channels, 10 patches, 9 classes, subject-disjoint), ISRUC (6 channels, 30 patches, 5 classes, subject-disjoint), TUEV (16 channels, 5 patches, 6 classes, record/prefix splits with unverified subject disjointness), SEED-V (62 channels, S = 1, 5 classes, within-subject trial splits, CBraMod only), and PhysioNet-MI (64 channels, 4 patches, 4 classes, probe comparison only, no matched token-wise pair). SEED-V has only one temporal patch, so only channel mixing is possible there. The authors note that CBraMod's released weights do not document whether TUEV was excluded from TUEG pretraining, so they do not treat CBraMod–TUEV results as evidence of transfer to an unseen dataset.

Why This Matters

Impact on research. The paper is a methodological caution: gains over a frozen-feature probe conflate adding capacity with adding the right structure. Many adapter papers implicitly claim that mirroring the backbone's inductive bias is beneficial; this study shows that claim needs a capacity-matched, protocol-controlled test, and that it can fail. It also introduces a reusable control design (branch-preserving MLP, sequence-encoder sensitivity follow-ups, class-level error localization) for EEG adaptation research.

Real-world applications:

• Clinical sleep staging — automated five-class sleep scoring from six bipolar channels, where the paper shows the adapter ranking is sensitive to whether the downstream sequence model is trained.
• Seizure and abnormal-event monitoring — TUEV-style event classification, where errors concentrate in specific classes such as GPED and eye-movement artifacts.
• Affect and mental-state decoding — FACED and SEED-V emotion recognition for brain–computer interface and neurotechnology products.
• Motor-imagery BCI — PhysioNet-MI, relevant to assistive communication and rehabilitation interfaces.

Industry relevance. Teams building on pretrained EEG backbones face a practical build-vs-buy decision for adapters: a simple parameter-matched MLP is cheap to implement, easy to maintain, and here often matches or beats a bespoke structured adapter. Knowing that no consistent benefit exists for the elaborate design is directly actionable for engineering budget and for benchmarking claims in product evaluations. Low absolute scores on LaBraM–FACED (.159–.195 balanced accuracy in the primary conditions) also flag how much headroom remains before deployment-grade claims.

Future Directions

• Isolate axis mixing from operator choice. The paper's RQ1 deliberately compares complete designs, so attention vs. MLP and token mixing vary together. The authors propose a parameter-matched axis-mixing MLP that retains cross-token interaction without attention to separate these factors.

• Disentangle the ISRUC protocol sensitivity. Training the downstream sequence encoder changes both its trainability and its dropout behavior; varying trainability while holding dropout and other settings fixed would show which is responsible.

• Test alternative feature arrangements and insertion points. The native adapter follows each backbone's axes and feature partitions, but alternative arrangements were not tested, nor whether CBraMod's final feature halves retain distinct channel and temporal roles. Adapters were only placed after the encoder, with shared optimization settings.

• Broaden initialization and evaluation coverage. Three seeds across six primary settings do not capture all initialization uncertainty; CBraMod–TUEV pretraining overlap is unresolved, LaBraM–TUEV uses a common probe reference with unverified independent replication, and held-out predictions limited the selector-sensitivity analysis.

Target Audience

Researchers and engineers working on EEG foundation models, parameter-efficient transfer learning, or biosignal representation learning who are designing or benchmarking downstream adapters. It is especially useful for those who need a template for capacity-controlled comparison, and for reviewers and practitioners who want to know whether structure-following adapters are worth the complexity before committing to them. Readers interested in the broader question of whether architectural inductive biases transfer across pretraining and adaptation stages will also find it relevant.

Authors’ abstract

Electroencephalography (EEG) foundation models learn representations that combine information across channels and time. This structure motivates adapters that model the same relationships during downstream learning. However, it is not clear whether such a design outperforms other candidate adapters. Moreover, in order to arrive at a fair comparison between candidate adapters, we also need to consider their trainable parameters. Here, we ask whether a backbone-informed (native) adapter makes better use of a given capacity by comparing it against a token-wise multilayer perceptron control that transforms each feature vector independently. We approximately match their trainable parameter counts and keep the pretrained encoder fixed. Across two EEG foundation models and tasks covering emotion recognition, sleep staging, and EEG event classification, the native adapter shows no consistent advantage. For one backbone, differences between the designs are small relative to variation across runs. For the other, the token-wise control performs better on emotion recognition and EEG event classification. On sleep staging, the native adapter's advantage over the token-wise control is reduced when we also train the component that processes sequences of EEG segments. The relative performance of the two adapters depends on the backbone, task, and downstream training protocol. These findings motivate evaluating backbone-informed adapters against capacity-matched alternatives.