When2What: A Timing-Conditioned Framework for Long-Horizon Forecasting of Continuous Cardiovascular Waveforms

AI summary

Overview

Research area: Machine learning for biomedical time-series forecasting — specifically long-horizon prediction of continuous cardiovascular waveforms (ECG, arterial blood pressure, photoplethysmogram).

Technical level: Advanced. The paper assumes familiarity with sequence models (Transformers, state-space models), autoencoder tokenization, custom loss functions, and event-detection evaluation metrics, and it reports architectural and training details at that level.

Scope (one sentence): The paper proposes When2What, a three-stage "Predict–Commit–Generate" framework that forecasts 60 seconds of continuous cardiovascular waveform from a 120-second history by explicitly predicting when future heartbeats will occur before generating what they look like.

What This Paper Is About

Cardiovascular waveforms are quasi-periodic signals built from recurring heartbeats, so a useful forecast must get two things right at once: the timing of each future beat and the local shape (morphology) of the waveform around it. Standard forecasters trained with pointwise mean squared error tend to average over uncertain future beat phases, which flattens sharp features like ECG R peaks and pulse upstrokes into featureless baselines over long horizons. The authors show that beat timing becomes unpredictable within seconds while beat shape stays stable over minutes, and they exploit that asymmetry by predicting discrete future beat times first, sharpening them into clean impulses, and then conditioning a waveform generator on those committed events.

Key Contributions

1. Timing-versus-morphology characterization: An analysis of intraoperative ECG showing that beat-time forecast skill from history decays within seconds due to stochastic heart-rate variability, whereas future beat morphology aligned at true beat times stays stable over several minutes — motivating a decoupling of the two problems.

2. A timing-conditioned forecasting framework: When2What, a Predict–Commit–Generate pipeline that turns a dense, decaying timing probability map into sharpened unit-height Gaussian impulses and conditions a neural generator on both those events and the observed history, producing 60-s multi-beat continuous waveforms without the peak flattening of direct regression.

3. Effectiveness and cross-signal generalization: Demonstrations that When2What outperforms existing time-series and physiological forecasters, transfers zero-shot to an external hospital cohort (MOVER) and an atrial fibrillation Holter cohort (SHDB-AF), and extends to ABP and PPG pulse waveforms.

4. Ablations and a trade-off analysis: Systematic ablations against unconditioned regression variants (Direct and Direct-MT), unsharpened timing maps, and static historical beat templates, plus controlled synthetic experiments showing that preserving sharp cardiac events under timing uncertainty mathematically requires trading off pointwise squared error.

Main Findings

• Beat timing decays fast; beat shape does not. On VitalDB intraoperative ECG, event skill drops rapidly within the first 10 s and reaches chance level (S = 0) after roughly 25 s. Exponential fits give decay constants of τ = 7.26 s for a simple RR extrapolation (median of the last five corrected beat intervals) and τ = 7.36 s for the learned When2What timing head. Over a 5-minute horizon, the median correlation between a patient's historical median beat template and future beats aligned at true R-peak times is 0.995 in the first 2 s and still 0.982 four to five minutes ahead.

• Strong long-horizon waveform performance. On the VitalDB ECG test cohort (2,286 windows from 845 patients), When2What reaches a 0–10 s waveform-channel event skill of 0.644, versus Informer at −0.069, HTF at −0.072, DLinear (100 Hz) at −0.061, PatchTST (100 Hz) at −0.096, and BeatLex at 0.152 — paired gains of +0.713, +0.716, +0.704, +0.739, and +0.491 respectively. Several direct forecasters score below zero, i.e., worse than chance.

• Competitive even on short horizons. For 2-s forecasts, When2What's 0–2 s event F1 is 0.833, compared with FADE at 0.782, PatchTST at 0.712, DLinear at 0.134, and HTF at 0.098. For 10-s-history to 10-s forecasts, it scores 0.644 event skill versus FADE at 0.085 and DLinear at −0.025.

• The explicit timing head outperforms RR-series forecasters. On the event channel, the When2What timing head achieves 0.672 event skill over 0–10 s, compared with TimesFM (500M, zero-shot) at 0.605, GBM with Huber loss at 0.615, GRU with Huber loss at 0.538, AR(8) at 0.522, PatchTST-RR at 0.416, and DLinear-RR at 0.264.

• Timing must be committed as explicit conditioning, not just supervised. A matched multi-task control (Direct-MT) with auxiliary timing supervision predicts future beat times accurately in its timing head, but its unconditioned waveform generator reproduces only 72.04% of those correctly predicted beats in the output waveform. When2What reproduces 95.62% and improves waveform event skill by +0.110 — evidence that feeding committed timing into the generator matters beyond an auxiliary loss.

• The commit/sharpening step is required. Feeding the raw, broadened timing map directly into the generator (the raw-map substitution in Section 4.3) produces smoothed, flattened heartbeats, which is why the non-neural Commit operator re-encodes selected events as unit-height Gaussians.

• The learned generator beats static templates. The learned generator outperforms static historical beat templates on event accuracy, 60-s waveform MSE, and resilience to perturbations of the event input.

• Methodological finding on evaluation metrics. Because low samplewise MSE does not guarantee detectable heartbeats, the authors pair pointwise error with forecast-verification skill scores using established beat detectors, and their synthetic experiments show that preserving sharp events under timing uncertainty inevitably increases pointwise MSE.

Note: the supplied paper text is truncated mid-sentence in Section 4.2. Details of Sections 4.3–4.5 (individual ablation tables, zero-shot transfer numbers, and the synthetic trade-off experiments) are therefore not reported here.

Methodology in Plain English

Setting up the problem. Each example gives the model 120 seconds of a single waveform sampled at 500 Hz (60,000 samples) and asks it to produce one continuous 60-second forecast (30,000 samples). Reference "beat events" are R peaks for ECG and maximal-upslope pulse anchors for ABP and PPG.

Measuring timing skill fairly. Predicted beats are matched to reference beats within ±50 ms. Because a faster heart rate makes random guesses more likely to land inside that window, the raw F1 score is rescaled against a chance baseline (a periodic pulse train at the window's true mean RR interval with random starting phase), giving a skill score S where 1 is perfect and 0 is random guessing.

Stage 1 — Predict. A learned timing predictor outputs a dense probability map over the 60-second horizon, marking likely beat locations. Because true beats are narrow (10 ms Gaussian bumps covering under 2% of the timeline), plain MSE would collapse to all zeros, so the timing loss combines a peak-weighted MSE with a differentiable soft-overlap F1 term.

Making it tractable. Instead of processing 90,000 raw time steps, a convolutional autoencoder compresses the 120-s history into 60 tokens (each covering 2 s), and 30 learnable horizon query tokens represent the 60-s forecast at the same 2-s resolution. These 90 tokens pass through a shared 4-layer, width-512 backbone that interleaves bidirectional Mamba state-space blocks with self-attention every second layer; the same encoder and backbone are shared by the predictor and the generator.

Stage 2 — Commit. The raw timing map broadens and weakens with lead time. A non-neural operator picks discrete beat times as local maxima above a validation-selected threshold, enforces a minimum refractory spacing of half the median of the last five eligible historical RR intervals, and replaces them with clean unit-height 10 ms Gaussian impulses.

Stage 3 — Generate. The waveform generator receives the sharpened map at two scales: as a token-level embedding added to the horizon queries (2-s scale, so the backbone knows the planned beat schedule), and as a full-resolution 500 Hz skip connection into the decoder output (millisecond anchoring of peaks and upstrokes).

Training trick. The generator is trained with teacher forcing on the ground-truth timing map, so its beat locations always align with the true beats in the target; otherwise pointwise losses would pressure it to smooth peaks. At inference, teacher forcing is removed and the committed map is used, with no future information.

Evaluation. Performance is scored on two channels: the event channel (discrete predicted beats versus reference beats) and the waveform channel (beats detected in the generated waveform by the NeuroKit2 detector versus reference beats), plus samplewise MSE over the full 60 s on z-scored waveforms. Uncertainty comes from patient-cluster bootstrap confidence intervals, with multiple random seeds per model and a held-out lockbox cohort used as a secondary test set.

Why This Matters

Impact on research. The paper reframes long-horizon physiological forecasting as a structured generation problem rather than a pointwise regression problem, and gives a concrete diagnosis of why direct MSE-trained forecasters flatten: the optimal point forecast is the conditional mean, which averages over phase-shifted beats. The normalized event-skill metric and the two-channel (event versus waveform) evaluation protocol also offer a template for judging whether a forecast contains usable beats at all, not just low average error.

Real-world applications:

• Intraoperative and critical-care monitoring: the data are intraoperative VitalDB recordings, where anticipating the next minute of ECG, ABP, and PPG could support anticipation of hemodynamic instability.
• Cross-hospital deployment without retraining: zero-shot transfer to MOVER (a separate medical center) and to SHDB-AF (ambulatory Holter with atrial fibrillation) suggests models can move between care settings and rhythms.
• Multi-signal bedside use: a single framework covering ECG, arterial blood pressure, and photoplethysmogram pulse waveforms fits the multi-parameter monitors already used in operating rooms and ICUs.
• Synthetic waveform generation and simulation: explicitly conditioned beat timing is directly relevant to physiological signal simulators and to augmenting scarce annotated clinical data.

Industry relevance. For medical-device and clinical-AI companies, the paper points at where forecast quality actually breaks in deployed monitors (beat timing uncertainty, not average error) and offers a lightweight architectural fix that shares one encoder and backbone between predictor and generator without increasing inference overhead. The focus on physically meaningful events rather than opaque sample error is also relevant to regulatory arguments about what a predictive monitor actually guarantees.

Future Directions

• Closing the timing gap. Timing skill still collapses to chance after roughly 25 s regardless of predictor, so the paper's own framing leaves open whether better use of long context, patient-specific history, or explicit uncertainty modeling can extend the usable timing horizon.
• Testing the timing–morphology asymmetry beyond cardiovascular signals. The design rests on an empirical observation in intraoperative ECG; whether the same seconds-versus-minutes asymmetry holds for other quasi-periodic physiology is an open question the paper does not settle.
• Handling irregular and pathological rhythms. Zero-shot results on the atrial fibrillation cohort are reported, but the framework's Gaussian-impulse event representation and refractory-spacing rule are built around regular beat structure, leaving room for event representations suited to highly irregular rhythms.
• Better objectives for event fidelity. The authors show that preserving sharp beats under timing uncertainty necessarily costs pointwise squared error; this invites continued work on metrics and training objectives that reward detectable, correctly placed events rather than average waveform agreement.

Target Audience

Machine learning researchers working on time-series forecasting and conditional generation, especially those interested in structured or event-conditioned generation and in state-space/Mamba backbones; biomedical engineers and clinical data scientists building predictive monitoring for ECG, blood pressure, or pulse oximetry; and clinical informatics groups evaluating whether a forecast contains usable physiological events rather than merely low average error. Readers without a background in sequence modeling will find the problem framing and the timing-versus-morphology intuition accessible, but the architectural and training details require intermediate-to-advanced familiarity.

Authors’ abstract

Long-horizon forecasting of quasi-periodic cardiovascular waveforms, such as the electrocardiogram (ECG), arterial blood pressure (ABP), and photoplethysmogram (PPG), requires predicting both when future cardiac events will occur (beat timing) and what the local structural pattern looks like around each event (beat morphology). However, standard sequence forecasters trained with pointwise mean squared error (MSE) average across uncertain future heartbeat phases, flattening sharp R peaks and pulse upstrokes into featureless baselines. Our analysis of intraoperative ECG reveals a clear temporal asymmetry: beat-time predictability from history decays within seconds due to stochastic heart-rate variability, whereas local beat morphology aligned at true beat times remains stable over several minutes. Guided by this insight, we introduce When2What, a timing-conditioned framework comprising three stages: Predict, Commit, and Generate. Given a 120-s history, When2What predicts a dense 60-s timing map, selects and sharpens discrete beat events into unit-height Gaussian impulses, and conditions a neural waveform generator on both these committed events and the observed history. Across intraoperative ECG, When2What substantially outperforms existing time-series and physiological forecasters, achieving a near-term waveform event skill of 0.644 and reproducing 95.62% of its predicted heartbeats in the generated waveform where direct long-horizon forecasters collapse into flat baselines. Furthermore, When2What transfers zero-shot to external hospital and atrial fibrillation cohorts, extends effectively to ABP and PPG pulse waveforms, and is supported by ablations and synthetic analyses that establish explicit timing conditioning as an effective paradigm for quasi-periodic long-horizon waveform forecasting.