The 2026 PNPL Competition: Word Classification and Efficient Cross-Subject Generalisation in LibriBrain100

AI summary

Overview

Research area: Non-invasive speech decoding from magnetoencephalography (MEG), benchmark design for machine learning, and brain-computer interfaces (BCIs).

Technical level: Intermediate. The paper is a competition description rather than a technical methods paper, so no deep neuroimaging or model-training background is required, but familiarity with classification metrics (balanced accuracy, recall@k), MEG terminology, and supervised/self-supervised learning helps.

Scope: This paper describes the design, data, tasks, metrics, baselines, rules, and schedule of the 2026 PNPL Competition, built on the LibriBrain100 MEG dataset, which adds a cross-subject generalisation track to the previous year's single-subject curriculum.

What This Paper Is About

The 2025 PNPL competition established benchmarks for non-invasive speech decoding using the LibriBrain dataset, which contained roughly 50 hours of MEG data from a single participant, and its winning entries reached F1-macro scores of 95.6% for speech detection and 73.6% for phoneme classification. That within-subject scale produces strong decoding, but a practical BCI must work for new users given only minutes of their data, not hours.

The 2026 competition responds by introducing LibriBrain100, an extended dataset with 32 additional subjects (roughly 40 minutes each) plus roughly 80 hours of within-subject data for Subject 0, and by advancing the task curriculum from speech detection and phoneme classification to word classification across two tracks: a Deep track for within-subject performance at scale, and a Broad track for cross-subject generalisation with progressively reduced subject-specific fine-tuning data (~40, ~20, and ~10 minutes).

Key Contributions

1. Word classification as a standardised competition task. Unlike phonemes, words form an effectively open set, and the choice of vocabulary affects measured performance. The paper addresses this by reporting three complementary evaluations — a dataset-tailored 50-word competition vocabulary, the Moses 50 vocabulary used in invasive studies, and open-vocabulary mutual information (OVMI) — while making top-10 balanced accuracy on the competition vocabulary the primary metric for leaderboards and prizes.

2. Cross-subject generalisation as an explicit competition target. The Broad track evaluates models that must adapt to new individuals with progressively limited data, and uniquely includes 8 holdout subjects with no released subject-specific training data at all, enabling evaluation of zero-shot cross-subject generalisation. The paper notes that previous state-of-the-art invasive systems were trained and evaluated on a single individual, so this regime has received little attention.

3. LibriBrain100 as a shared competition resource. The competition is the first built around a large-scale, multi-subject MEG dataset, combining the deepest within-subject MEG dataset recorded to date (~80 hours) with 32 additional subjects, allowing within-subject and cross-subject benchmarking in a single competition.

4. Baselines, code, and onboarding materials. The paper provides two reference models (one per track), a pnpl Python library for automatic data download and PyTorch loading, three interactive Google Colab tutorials, standardised train/validation/test splits, a public leaderboard, and a Kaggle submission platform.

Main Findings

• Prior competition scale: The 2025 PNPL competition attracted 155 registered teams from 15 countries and generated 6,041 submissions. By April 2026, the LibriBrain dataset had reached over 16,000 downloads.

• Dataset composition: LibriBrain100 comprises non-invasive MEG recordings from 33 subjects listening to naturalistic speech. Subject 0 is extended to roughly 80 hours, the deepest within-subject MEG dataset recorded to date. For an additional 32 subjects, roughly 40 minutes of MEG data were collected, released in tiers: the full ~40 minutes for 12 subjects, ~20 minutes for 10, and ~10 minutes for a further 10.

• Zero-shot cohort: For a final group of 8 subjects not included in LibriBrain100 (subjects 33–40), no subject-specific training data are available, requiring zero-shot cross-subject generalisation. The full data for subjects 1–32 will be released after the competition.

• Deep track baseline: The d'Ascoli reference model scored BAcc@1 of 25.60%, BAcc@10 of 73.23%, and OVMI of .220 on the competition vocabulary, and BAcc@1 of 38.84%, BAcc@10 of 83.60%, and OVMI of .157 on the Moses vocabulary. Random chance is 2.00% BAcc@1, 20.00% BAcc@10, and .000 OVMI.

• Broad track baselines: MEG-XL scored BAcc@1 of 6.20%, BAcc@10 of 42.96%, and OVMI of .014 on the competition vocabulary, and 10.21%, 52.23%, and .011 on the Moses vocabulary. The d'Ascoli model scored 5.14%, 33.13%, and .009 on the competition vocabulary, and 6.10%, 49.56%, and .006 on the Moses vocabulary.

• Reference-model rationale: With large amounts of within-subject data, supervised models outperform self-supervised ones, so the Deep track uses the supervised word decoding model of d'Ascoli et al. (2025). With limited per-subject data, pre-trained self-supervised priors confer a substantial advantage, so the Broad track uses MEG-XL, a self-supervised foundation model pre-trained on roughly 300 hours of MEG from 800 subjects.

• No competition results reported: Because the competition runs from 15 July to 15 October 2026, the paper reports only reference baselines and random-chance scores; participant results are not reported.

• Vocabulary design trade-offs: Custom vocabularies such as the most-frequent K words in a corpus maximise training examples and reported accuracy but are not comparable across studies and are often dominated by function words of limited communicative utility. Standardised vocabularies such as Moses 50 enable comparisons, including with invasive BCIs, but may be poorly attested in a given corpus. OVMI addresses both concerns by jointly accounting for decoding accuracy and corpus coverage.

Methodology in Plain English

The task is word classification from MEG: given a window of brain data with 306 sensors across T time samples, predict which word from a fixed 50-word vocabulary the participant is hearing. Neural information relevant to the task may include phonetic representations from auditory and motor cortices and lexical semantic representations that cover most of the cortex — the latter being an argument for MEG's whole-brain coverage over surgically implanted arrays that sample only a limited cortical region.

Evaluation uses top-10 balanced accuracy with K=50, which asks whether the correct word appears anywhere in the model's top-10 predictions, averaged so each word contributes equally regardless of how often it appears in the evaluation set. Uniform random guessing yields 20% expected BAcc@10 for a 50-word vocabulary. Because BAcc@10 does not distinguish a first-place from a tenth-place correct answer, top-1 balanced accuracy is also computed to break ties. Open-vocabulary mutual information, an information-theoretic measure combining vocabulary coverage under a reference distribution (SUBTLEX-UK, reported in bits per word perceived) with in-vocabulary mutual information between intended word and decoder output, is reported as an auxiliary metric.

Data are released in tiers so that participants must predict with decreasing amounts of subject-specific data. For the Broad track fine-tuning baseline, half of the data from the Sherlock 1 Session 11 recording was used for fine-tuning and the other half for validation across Subjects 1–32, while the Deep track used all available training data for Subject 0. Evaluation happens on a separate holdout set whose recordings are released but whose labels are withheld; the holdout is split into one partition for continuous leaderboard updates and one reserved for final ranking, mirroring the 2025 design to prevent overfitting to the holdout distribution.

Why This Matters

Research impact. Progress in invasive speech BCIs has been rapid — vocabulary sizes have grown from 50 words to over 125,000 words, and word-error rates have fallen to 2.5%, which is below the roughly 5–10% reported for human transcription on standard ASR benchmarks. Non-invasive decoding has advanced more slowly and been harder to compare across studies because of missing shared infrastructure: common datasets, fixed evaluation splits, standard metrics, and baseline implementations. This competition supplies that infrastructure for word classification, and its explicit reporting of target vocabulary addresses a documented source of incomparability between studies.

Real-world applications.

• Restoring communication for people living with profound paralysis, without the risks of brain surgery.
• Clinically feasible BCI calibration, since the ~10-minute fine-tuning regime falls within a clinically feasible range.
• Assistive communication vocabularies designed with patient input, such as Moses 50, which emphasises words for expressing needs and feelings.
• Standardised benchmarking infrastructure that lets laboratories compare decoding approaches directly rather than on incomparable custom vocabularies.

Industry relevance. The competition targets data-efficiency and cross-subject generalisation, which are the constraints that determine whether a decoding system can be deployed at all rather than only demonstrated in a research setting. It is also backed by a self-supervised foundation model (MEG-XL) and by involvement from Google DeepMind among the authors, and it builds on a clinical framing that non-invasive alternatives are "the real prize" given that surgery carries inherent risks and does not easily scale across patients.

Future Directions

• Joint training across subjects. The authors suspect that training jointly on Subject 0 and Subjects 1–32 will improve generalisation on the latter subjects further, but leave this and other avenues to competition participants.
• Closing the gap to open-vocabulary brain-to-text. Open-vocabulary B2T from speech-related neural activity remains out of reach for current non-invasive systems, and word classification serves as an intermediate benchmark on which to measure progress in representation learning, cross-subject generalisation, and decoding under non-invasive constraints.
• Extending the multi-year curriculum. The competition series is framed as a curriculum progressing from foundational tasks (speech detection, phoneme classification) toward the linguistic complexity required for a practical BCI, with word classification positioned as a step toward that goal.
• Post-competition data release and analysis. The full data for subjects 1–32 will be released after the competition, and the organising team independently conducts post-competition analysis of the results, which may inform the next edition.

Target Audience

This paper is most useful to machine learning researchers who may have no prior experience analysing neural data and are considering entering the competition, since the tracks are explicitly designed to be accessible without domain-specific knowledge or specialised hardware. It is also relevant to MEG and speech-decoding researchers looking for standardised benchmarks and evaluation protocols, and to BCI practitioners interested in what performance is achievable under clinically feasible data constraints. Finally, it serves as a reference for anyone who wants to compare competing decoding approaches on a common vocabulary, metric, and data split, or who needs to cite the composition of the LibriBrain100 dataset and its two reference baselines.

Authors’ abstract

The ambition of the 2025 PNPL competition (Landau et al., 2025) was to launch a multi-year curriculum for non-invasive speech decoding. Designed to progress from foundational tasks toward the linguistic complexity required for a practical brain-computer interface (BCI), it set the stage with speech detection and phoneme classification tasks. Winning submissions reached F1-macro scores of 95.6% and 73.6% on the respective tasks (Elvers et al., 2026), highly significant advances. This success was built on the LibriBrain dataset (Özdogan et al., 2025), the largest within-subject MEG dataset recorded at the time with ${\sim}50$ hours of data for one subject. However, while within-subject scale drives strong decoding performance, a practical BCI must generalise to new users from minutes of data, not hours. The 2026 PNPL competition responds to this challenge with LibriBrain100 (Mantegna et al., 2026), an extended LibriBrain dataset with 32 additional subjects (${\sim}40$ minutes each) plus even more within-subject data (${\sim}80$ hours). Advancing the curriculum of tasks to focus on word classification, two complementary tracks are presented in this competition: the Deep track targets within-subject word classification at scale, aiming at the best possible performance; the Broad track targets cross-subject generalisation, progressively reducing the amount of subject-specific fine-tuning data from ${\sim}40$ to ${\sim}20$ to ${\sim}10$ minutes, a duration that falls within a clinically feasible range and brings us a step closer to a non-invasive BCI capable of restoring communication to people living with profound paralysis.