PhysioWave-T: Token-Efficient Wavelet Foundation Models for Biosignals

AI summary

Overview

Research area: Biomedical signal processing and self-supervised foundation models for physiological time series (EEG, ECG, sEMG), with an emphasis on token-efficient multiscale modeling and multimodal fusion.

Technical level: Advanced. The paper assumes familiarity with wavelet decompositions, Transformer tokenization, masked self-supervised pre-training, and multimodal fusion.

Scope: A single paper introducing PhysioWave-T — a family of wavelet-based foundation models with a "dynamic scale folding" tokenizer, route-specific front ends for heterogeneous acquisition setups, an EEG encoder scaled to 15.4M pre-training windows, and a reliability-aware fusion module for combining EEG, ECG and sEMG.

What This Paper Is About

Physiological signals come from wildly inconsistent hardware: EEG ranges from 2-channel wearables to 128-channel scalp arrays, ECG from single-lead patches to 12-lead clinical systems, and sEMG from wristbands to 64-channel high-density grids, each at its own sampling rate. Wavelet analysis captures the multiscale structure of these non-stationary signals well, but exposing every wavelet subband to a Transformer multiplies the token count by the number of decomposition levels plus one — a cost that explodes for dense electrode arrays. The paper's goal is to keep the multiscale wavelet information while shrinking the token sequence, to let one Transformer backbone serve many acquisition configurations, and to fuse modalities whose reliability varies from recording to recording.

Key Contributions

1. Dynamic scale folding (token-efficient multiscale modeling). Instead of tokenizing each of the J+1 wavelet subbands separately, the front end recombines them into a single signal with the shape of the input, compressing the sequence from (J+1)CS to CS tokens for J levels, C channels and S patches. The authors quantify the effect on accuracy, latency, memory and computation.
2. An EEG foundation model spanning acquisition configurations. Route-specific wavelet front ends plus a learned, extensible channel-name vocabulary let one shared 30M-parameter backbone learn from montages of 19 to 128 channels at 256 and 512 Hz, and transfer to unseen configurations — including 2-channel wearable EEG — by attaching a lightweight new front end.
3. Reliability-aware latent fusion (RALF). A fusion module that consumes four summary tokens per independently pre-trained encoder, predicts a per-sample reliability score for each modality, and uses that score both to gate attention and to weight a residual contribution from each unimodal prediction, with explicit masking for missing modalities.
4. A unified biosignal model family. Alongside the 30M EEG encoder, the paper pre-trains a 30M ECG encoder and a 12M sEMG encoder under the same objective, and reports state-of-the-art results across the three modalities.

Main Findings

• Token, latency and memory savings: Across four EEG routes, folding reduces Transformer latency by 4.0–7.2x and peak memory by 6.8–12.7x relative to scale expansion without accuracy loss, cutting per-batch memory on 128-channel recordings from approximately 32 GB to 2.5 GB. For the three-level EEG decomposition the token reduction is fourfold, and the quadratic attention term shrinks by (J+1)².
• EEG benchmark leadership: The fine-tuned EEG encoder achieves the highest value on eight of nine metrics against BENDR, BIOT, LaBraM and EEGPT across Sleep-EDFx, PhysioNet-P300 and KaggleERN.
• Sleep staging despite unseen inputs: On Sleep-EDFx, whose Fpz-Cz and Pz-Oz derivations at 100 Hz match no pre-training route or vocabulary entry, PhysioWave-T raises kappa from 0.686 to 0.737 and weighted F1 from 76.5% to 80.9% over EEGPT. The scratch model alone reaches kappa 0.736, indicating the multiscale front end itself suits band-limited sleep rhythms.
• Error-related negativity: On KaggleERN, balanced accuracy improves by 3.3 points and kappa by 0.018 over EEGPT — the largest balanced-accuracy gain among the three tasks.
• P300 needs little adaptation: On PhysioNet-P300, balanced accuracy and AUROC each rise by 2.0 points while kappa stays within 0.003 of EEGPT; the frozen encoder already exceeds every baseline in balanced accuracy and AUROC, and fine-tuning adds only 0.9 points.
• Adaptation modes matter differently: On KaggleERN, probing falls 3.4 points below scratch, yet fine-tuning exceeds scratch by 4.7 balanced-accuracy points.
• Label efficiency: Using only 10% of KaggleERN labels, the pre-trained encoder surpasses a fully supervised scratch model.
• Wearable seizure detection: On two-channel behind-the-ear seizure detection, the encoder reaches 61.4% sensitivity at ≤11 false alarms per hour, outperforming LUNA-L while using 25.4M versus 40.4M parameters.
• Multimodal fusion on SeizeIT2: RALF raises window-level AUPRC from 0.321 to 0.399 over the best unimodal model and outperforms weighted-logit and concatenation fusion in AUPRC across all six training variants. For event sensitivity at 11 false alarms per hour, the abstract reports an increase from 61.4% to 74.4% with RALF, while the introduction reports 61.4% to 78.9% — the paper states both figures.
• Robustness: RALF retains an AUPRC of at least 0.33 under every tested single-modality corruption.
• ECG and sEMG state of the art: PhysioWave-T sets new state-of-the-art sEMG accuracies and the highest macro-F1 on PTB-XL Super and Chapman–Shaoxing.
• Interpretable structure emerges: Channel embeddings recover scalp topography without coordinate supervision, and the slowest wavelet scale's folding weight progressively increases from wakefulness to deep sleep.
• Reconstruction quality: With 70% of patches masked, masked-patch correlations range from 0.79 to 0.89 for the raw-EEG head and 0.75 to 0.88 for the folded-wavelet head across all four EEG routes; single-channel correlations reach 0.93 and 0.96 on the selected M3CV and HBN channels, while the selected TUEG channel is harder (r = 0.52, though its window-level correlation is 0.79).

Methodology in Plain English

The design principle is that everything tied to a particular sensor setup is handled before tokenization, and everything about learning physiology happens after it.

Routes. Each recording is assigned a "route" defined by its channel count and sampling rate, read from metadata rather than predicted. Each route gets its own wavelet front end, and each sampling rate gets its own patch projection. The channel vocabulary, position code and Transformer are shared across all routes of a modality. Patches are defined by physical duration, so a token always spans the same amount of time regardless of sampling rate.

Learnable wavelet analysis. Following the authors' earlier PhysioWave work, each front end keeps a bank of learnable wavelet filter pairs initialized from named wavelets such as db6 and sym4. At each level a small network looks at the current signal approximation and predicts softmax mixture weights over the filter bank, so the basis adapts to the input. The approximation and detail subbands are then blended through per-channel gates predicted by attention over pooled channel statistics, which replaces a fixed skip connection with a learned trade-off. A cross-scale feed-forward block lets each level attend to the refined maps of all shallower levels.

Dynamic scale folding. This is the key new piece. The J+1 subbands are averaged to form a reference signal, and a residual branch adds a weighted sum of each subband passed through a short depthwise synthesis filter. The weights are predicted per channel and per patch-duration time block from three statistics of each subband — log RMS amplitude, entropy of normalized squared amplitudes, and log-one-plus-kurtosis — plus a learnable scale prior, with a KL penalty toward uniform weights so no single band dominates. Filters are initialized to a delta and the scalar mixing factor to 0.1, so the fold starts as an exact average. Because the folding weights, filters and prior carry no channel index, folding transfers unchanged across montages.

Tokenization. The folded signal is cut into fixed-duration patches, projected by the sampling-rate-specific embedding, and augmented with a learned channel-name embedding (gated by a scalar that starts at zero) and a two-dimensional sinusoidal position code, with rotary position embeddings inside attention.

Pre-training. Masking acts on the raw input, not just the tokens, because convolutional and cross-scale mixing would otherwise leak hidden content through neighboring features. Each window produces a clean view (which supplies targets and mask-selection scores) and an online view with selected raw patches zeroed before the front end. Selection follows the earlier frequency-guided strategy: spectral importance e combined with uniform noise z as s = 0.6e + 0.4z, masking the top 70%. Two shallow decoders per sampling rate reconstruct the clean folded-wavelet representation (stop-gradient, normalized per patch) and the raw signal, with the loss weighting the two reconstruction terms equally plus a 10⁻³ KL term on the folding weights. Corpora are sampled with probability proportional to n_d^0.5 to stop large datasets from dominating, and per-device batch sizes scale inversely with route token count.

Adaptation. Three modes are compared: full fine-tuning with layer-wise learning-rate decay, probing with transferred components frozen, and training the same architecture from scratch.

Fusion. Each frozen or adapted encoder is reduced to four summary tokens by an attention pool with four learned queries. Eight fusion tokens attend to their concatenation across two cross-attention layers. A sigmoid head predicts each modality's reliability, which gates its attention keys and values and contributes a normalized residual from its unimodal logits. During training, modalities are corrupted with probability 0.5 using noise, baseline drift, saturation, bad channels or temporal dropout, and the sampled severity defines the reliability target (1 − σ). A symmetric KL term encourages predictions on random modality subsets to agree with the full-set prediction.

Why This Matters

The paper reframes the bottleneck in biosignal foundation models: rather than choosing between multiscale analysis and tractable sequence length, it shows that the two can be decoupled by folding subbands before tokenization. Equally important, it demonstrates a clean separation between acquisition-specific processing and shared representation learning, which is what allows a single backbone to serve 2-channel wearables and 128-channel clinical arrays without discarding, padding or resampling channels. The reported memory reduction — roughly 32 GB down to 2.5 GB per batch on 128-channel EEG — is the kind of change that moves dense-EEG foundation models from multi-GPU clusters toward practical training budgets.

Real-world applications suggested by the evaluated tasks:

• Wearable seizure monitoring: two-channel behind-the-ear EEG alone, and EEG+ECG+EMG fusion on SeizeIT2, for event detection at a fixed false-alarm rate.
• Sleep staging: five-stage classification on Sleep-EDFx, including derivations at sampling rates unseen during pre-training.
• Clinical ECG interpretation: multi-label ECG tasks using PTB-XL Super and Chapman–Shaoxing.
• Brain–computer interfaces and ergonomics: P300 target detection, error-related negativity detection, and sEMG for wristband typing (emg2qwerty) and hand pose estimation (emg2pose).

Industry relevance: the route-and-front-end design maps directly onto the reality of commercial device fleets, where every product revision changes electrode count or sampling rate; adding a new sensor configuration requires only a lightweight front end and a new vocabulary entry rather than retraining a backbone. Token efficiency and the reported peak training memory (26 GB for 128-channel EEG, 19 GB for ECG, 16 GB for 64-channel sEMG on A100 GPUs) are relevant to teams weighing on-premise versus cloud training, and the reliability weighting addresses the practical problem of one failing sensor degrading a multimodal wearable decision.

Future Directions

• Joint multimodal pre-training. The three encoders are trained independently and only fused afterwards with frozen encoders in the seizure experiments; whether shared cross-modal pre-training improves on this is left open.
• Resolving and reconciling the reported fusion gains. The abstract and introduction give different event-sensitivity figures for RALF at 11 false alarms per hour (74.4% versus 78.9%), and a full ablation of which corruption types drive the reliability estimate most would sharpen the claims.
• Scaling routes and modalities further. The vocabulary already extends to ECG leads and sEMG sites, and unseen downstream configurations are handled by new front ends; how far this transfers to configurations far outside the pre-training routes (for example rates below 100 Hz, which appear only in downstream Sleep-EDFx data) remains to be measured.
• Deployment under hardware constraints. The efficiency gains are reported as latency and memory at the Transformer level; on-device inference, quantization and energy cost on wearable hardware are not reported and are natural next steps.

Target Audience

Researchers and engineers working on biosignal foundation models, self-supervised learning for time series, and multimodal physiological sensing. It is also relevant to applied teams building wearable or clinical monitoring products who need one model to serve heterogeneous sensor configurations, and to readers interested in learnable wavelet representations as an alternative to fixed spectral tokenization. Readers without background in wavelet analysis or Transformer pre-training will find the architecture sections dense, though the motivation and results sections are accessible.

Authors’ abstract

We present PhysioWave-T, a token-efficient wavelet foundation model family for EEG, ECG, and sEMG. Dynamic scale folding recombines J+1 wavelet subbands prior to tokenization, compressing sequence length from (J+1)CS to CS tokens for J levels, C channels, and S temporal patches. Route-specific front ends and learned channel-name embeddings allow one Transformer per modality to handle diverse montages and sampling rates, while unseen setups reuse the backbone via lightweight adapters. We pre-train a 30M-parameter EEG model on over 15M windows (19--128 channels), alongside ECG (30M) and sEMG (12M) encoders. Across four EEG routes, scale folding cuts Transformer latency by 4.0--7.2x and peak memory by 6.8--12.7x over scale expansion without accuracy loss. Our EEG encoder leads 8 of 9 metrics against BENDR, BIOT, LaBraM, and EEGPT, and surpasses LUNA-L on wearable seizure detection with fewer parameters. Reliability-aware latent fusion (RALF) further integrates summary tokens from independently pre-trained encoders via sample-adaptive weights. On SeizeIT2, RALF raises event sensitivity at 11 false alarms/h from 61.4% to 74.4% and window AUPRC from 0.321 to 0.399 over the best unimodal baseline, outperforming weighted-logit and concatenation fusion across all variants. PhysioWave-T also achieves state-of-the-art sEMG accuracies and top macro-F1 on PTB-XL Super and Chapman-Shaoxing. Code and models: https://github.com/ForeverBlue816/PhysioWave.