EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models

AI summary

Overview

Research area: Machine learning for brain-computer interfaces — specifically Electroencephalography (EEG) foundation models, self-supervised representation learning, and benchmarking methodology.

Technical level: Advanced. The paper assumes familiarity with Transformers, self-supervised pre-training objectives, transfer learning, and BCI signal processing, though its central conclusions are stated plainly enough for a broader audience.

Scope: A systematic review of 55 EEG foundation models combined with a controlled benchmark of 12 open-source models across 13 datasets and 9 BCI paradigms, evaluating cross-subject generalization, few-shot adaptation, and scaling behavior.

What This Paper Is About

EEG foundation models promise to learn general-purpose brain-signal representations that transfer to many downstream tasks, mirroring the success of large language and vision models. However, existing EEG FMs were each evaluated with different preprocessing, pre-training objectives, and downstream protocols, so no fair comparison existed. This paper asks three questions: whether these models truly learn transferable representations, whether they actually beat conventional models trained from scratch, and whether bigger models or more pre-training data produce better results.

Key Contributions

1. A unified taxonomy of 55 EEG foundation models. The authors survey the largest collection of EEG FMs to date and organize them into a coherent design space spanning data standardization (channel unification, resampling, normalization/alignment), backbone architectures, and self-supervised pre-training strategies (masked raw-signal reconstruction, masked token reconstruction, frequency-domain reconstruction, codebook-based objectives, and autoregressive pre-training).

2. A fair and comprehensive benchmark. Twelve open-source EEG FMs are evaluated alongside competitive specialist baselines — traditional machine learning, CNN-based, and Transformer-based models trained from scratch — on 13 datasets covering nine BCI paradigms (motor imagery, P300, SSVEP, clinical detection, emotion recognition, visual decoding, fatigue detection, sleep staging, and mental workload).

3. Two realistic evaluation scenarios. Beyond the standard leave-one-subject-out (LOSO) protocol, the paper introduces a within-subject few-shot setting using roughly 1/20 to 1/100 of the labeled data typically used in LOSO, reflecting rapid calibration conditions in real deployments.

4. A direct test of transferability and scaling. The authors compare full-parameter fine-tuning against linear probing to measure how transferable pre-trained features really are, and they examine whether larger parameter counts and larger pre-training corpora lead to better downstream generalization.

Main Findings

• Linear probing is frequently insufficient. Freezing the pre-trained encoder and training only a classification head often underperforms full-parameter fine-tuning. This suggests the learned representations are not immediately linearly separable for many BCI tasks and require substantial task-specific adaptation to be useful.

• Specialist models trained from scratch remain competitive. Conventional CNN and Transformer architectures trained only on the downstream fine-tuning data hold their own against fine-tuned EEG foundation models across many tasks. The presumed advantage of large-scale pre-training does not consistently materialize.

• Larger models do not necessarily generalize better. Under current data regimes and training practices, increased model size does not reliably translate into improved downstream performance, and scaling behavior for EEG FMs does not mirror the clean scaling laws observed in language and vision.

• The field is growing rapidly but unevenly. Of the 55 surveyed models, 16.4% appeared in 2024 and 67.3% in 2025–2026. Transformer backbones are used by roughly 83.6% of models, random masking is the dominant strategy (~71.7%), and raw-signal reconstruction is the most common target (~25.5%).

• Model scales span an enormous range. Parameter counts vary from under one million to several billion, and pre-training corpora range from about 1,000 hours to over 61,000 hours, with training budgets varying from a single GPU to hundreds.

• The design space is fragmenting. Most models aim for general-purpose representations, but a substantial subset targets a single paradigm (e.g., motor imagery, epilepsy, emotion, sleep), trading cross-paradigm generalizability for domain-aligned pre-training.

• EEG data preprocessing is highly inconsistent. Channel unification follows six distinct strategies (fixed montage, canonical template remapping, channel-wise modeling, learnable channel embeddings, coordinate-aware spatial encoding, and projection to a unified latent space), and only 50% of studies resample to a common 200 Hz, complicating cross-model comparison.

• Data augmentation for EEG pre-training is underexplored. Only a handful of surveyed models explicitly incorporate augmentation, and the policies are heterogeneous, pointing to an open design area.

Methodology in Plain English

The authors combined a literature survey with a controlled empirical study.

First, they read and catalogued 55 EEG foundation models, extracting for each one the pre-training data scale, preprocessing pipeline, architecture, pre-training objective, parameter count, and computational cost. They then grouped these choices into a unified framework so that models could be compared on common axes rather than on their authors' own terms.

Second, they selected 12 open-source EEG FMs and evaluated them on 13 public datasets covering nine BCI paradigms. Every model was tested under the same two conditions: leave-one-subject-out, where training data from multiple subjects is pooled and the model is tested on a held-out subject, and within-subject few-shot adaptation, where only a small amount of labeled data from the target subject is available. For each model, they compared full-parameter fine-tuning with linear probing so that they could separate quality of the pre-trained features from the benefit of task-specific tuning.

Third, they compared these results against strong baselines trained from scratch — traditional machine learning pipelines, CNN architectures such as EEGNet and ShallowConvNet, and Transformer models — to establish whether pre-training adds real value. Finally, they grouped models by parameter count and pre-training corpus size to check whether performance improved with scale.

Why This Matters

Impact on research. The paper provides the first controlled comparison of EEG foundation models under a shared protocol, and its negative results are consequential: they challenge the assumption that large-scale self-supervised pre-training automatically yields superior EEG decoders. This pushes the field toward careful ablation, honest baselines, and physiologically motivated design rather than scale alone.

Real-world applications.

• Assistive communication for severe motor impairment — BCIs that let people with amyotrophic lateral sclerosis, brainstem stroke, or high spinal cord injury spell or control devices, where fast per-user calibration matters more than benchmark accuracy.
• Clinical monitoring and diagnosis — epilepsy seizure detection, sleep staging, and assessment of disorders of consciousness, where cross-patient variability is severe and labeled data are scarce.
• Cognitive and mental state monitoring — workload estimation, fatigue detection, and emotion recognition for safety-critical work environments such as aviation and driving.
• Consumer and portable neurotechnology — heterogeneous consumer EEG headsets with different electrode layouts, which demand models that tolerate channel variability and short calibration sessions.

Industry relevance. EEG hardware vendors, neurotechnology startups, and medical device companies need models that generalize across devices and calibrate quickly for new users. This paper's finding that linear probing is often insufficient and that specialist models stay competitive directly informs product decisions about whether to invest in large pre-trained backbones or lean on lightweight task-specific models.

Future Directions

• Design physiologically grounded augmentation policies. Current augmentation for EEG FMs is sparse and ad hoc. Policies that exploit oscillatory structure, spatial topography, and inter-trial variability could introduce meaningful invariances while preserving task-relevant information.

• Move beyond linear probing. Since frozen features are frequently inadequate, open questions remain about how to build encoders whose representations transfer through prompting or lightweight adaptation rather than expensive full fine-tuning.

• Explain the missing scaling laws. It is unclear whether the absence of scaling benefits stems from insufficient or noisy pre-training data, ineffective objectives, architectural mismatches, or simply the low signal-to-noise ratio of scalp EEG. Identifying the bottleneck is a central open problem.

• Standardize evaluation and unify heterogeneous montages. Consistent preprocessing, resampling, and protocol reporting would make cross-model comparison meaningful, while coordinate-aware or latent-space channel unification could allow one model to serve genuinely diverse devices.

• Clarify when paradigm-specific pre-training wins. Since the target paradigm is often known before downstream data collection, the trade-off between general-purpose and paradigm-specific models deserves systematic study rather than anecdotal treatment.

Target Audience

This paper is most valuable to machine learning and BCI researchers developing or evaluating EEG foundation models, including graduate students entering the field who need a structured map of a fast-moving literature. It is also directly relevant to applied scientists and engineers in neurotechnology and medical device companies who must decide whether foundation-model pre-training justifies its computational cost, and to benchmark designers and reviewers who need principled protocols for comparing self-supervised models on physiological time-series data.

Authors’ abstract

Electroencephalography (EEG) foundation models (FMs) have recently emerged as a promising paradigm for brain-computer interfaces, aiming to learn transferable neural representations from large-scale heterogeneous recordings. Despite rapid progress, a fair and comprehensive comparison of existing EEG FMs is still lacking, owing to inconsistent pre-training objectives, preprocessing choices, and downstream evaluation protocols. To fill this gap, we present EEG-FM-Compass. We first review 55 representative models and organize their design choices into a unified taxonomic framework including data standardization, model architectures, and self-supervised pre-training strategies. We then evaluate 12 open source FMs and competitive specialist baselines across 13 EEG datasets spanning nine brain-computer interface paradigms. Emphasizing real-world deployments, we consider both cross-subject generalization under a leave-one-subject-out protocol and rapid calibration under a within-subject few-shot setting. We further compare full-parameter fine-tuning with linear probing to assess the transferability of pre-trained representations, and examine the relationship between model scale and downstream performance. Our results indicate that: 1) linear probing is frequently insufficient; 2) specialist models trained from scratch remain competitive across many tasks; and 3) larger FMs do not necessarily yield better generalization performance under current data regimes and training practices.