Label-Efficient Deep Learning for ECG Delineation: A Multi-Dataset Benchmark against Widely Used Delineation Tools

AI summary

Overview

Research area: Machine learning for biomedical signal processing, specifically deep learning-based electrocardiogram (ECG) delineation, evaluated with label-efficient learning strategies (self-supervised pretraining and semi-supervised learning) and benchmarked against established rule-based and commercial delineation tools.

Technical level: Intermediate (readers will benefit from familiarity with semantic segmentation, self-supervised learning objectives, and standard ECG interval terminology).

Scope in one sentence: This paper asks whether self-supervised and semi-supervised training can produce a deep learning ECG delineator that beats widely used open-source and commercial tools, and answers it through a two-stage study across one internal and four external datasets.

Note on completeness: the supplied paper text is truncated. It ends mid-sentence in Section VI-D, so the final part of the Discussion and the Conclusion are not available, and the Supplementary Material (Sections VIII–XIV) referenced throughout, including the reported statistical test details and several tables (S8–S14, Figs. S1–S3), is not included in the content analyzed here.

What This Paper Is About

ECG delineation means locating the boundaries of the P wave, QRS complex, and T wave in a heart signal so that clinically used measurements such as the PR interval, QRS duration, and QT interval can be derived. Deep learning can do this accurately, but it normally requires large amounts of expensive, expert-annotated, time-point-level labels, and it has never been directly compared head-to-head with the rule-based open-source tools and commercial packages that clinicians and researchers actually use. This paper tests whether label-efficient training (pretraining on unlabeled ECGs plus semi-supervised fine-tuning) solves the annotation problem, and then benchmarks the resulting model against NeuroKit2, ECGdeli, Prominence, and CalECG across multiple datasets.

Key Contributions

1. Systematic, multi-dataset evidence that self-supervised pretraining helps ECG delineation. The authors compare three pretraining objectives (MAE, MoCo, MERL) crossed with supervised versus semi-supervised fine-tuning across one internal test set (the QTDB + ISP test splits) and four external test sets (LUDB, Zhejiang, RDB, mECGDB).

2. The first benchmark of a label-efficiently trained deep learning delineator against multiple open-source libraries and a commercial package. The comparison uses three complementary metrics (mIoU, averaged interval error, and averaged point-wise sensitivity), plus point-wise sensitivity, positive predictive value, and localization error analyses.

3. Rhythm-stratified analysis across diverse external datasets. Using RDB, which contains delineation labels across eight rhythm types, the authors quantify performance separately on sinus and arrhythmia groups and measure how much each method degrades between them.

4. Characterization of the distinctive behavior of each tool. The paper shows the deep learning model achieves the best overall performance while remaining the most robust under arrhythmia, and reports the specific fiducial points at which each baseline fails, yielding practical guidance on tool selection.

Main Findings

• Without pretraining, semi-supervised learning beats purely supervised learning. mIoU improves on all datasets, from 63.5% to 71.4% on average. The largest gain is on Zhejiang, where mIoU rises from 53.3% to 66.0%; because Zhejiang consists entirely of recordings with arrhythmic beats, the authors read this as evidence that the semi-supervised benefit is pronounced under arrhythmic conditions. Averaged interval error also improves on all datasets except RDB, where supervised learning gives a lower error.

• Pretraining helps, but the objective matters. MAE and MoCo significantly outperform training without pretraining; MERL does not. Averaged across datasets under semi-supervised fine-tuning, MERL reaches mIoU 68.7% versus 75.2% for MAE and 75.0% for MoCo. On the internal test set the objectives are near saturation (mIoU 82.2–82.8% for MAE and MoCo; 81.0% for MERL under semi-supervised fine-tuning), and the differences widen on the external sets. The authors note MERL is multimodal and contrastive, but since MoCo is also contrastive, they suggest the gap may come from aligning ECGs to text, which could encourage invariance to the fine morphological detail delineation depends on.

• The two effective objectives interact differently with the fine-tuning strategy. MAE performs comparably under supervised and semi-supervised fine-tuning (mIoU 75.3% vs. 75.2%), whereas MoCo benefits substantially from semi-supervised fine-tuning (mIoU 72.4% vs. 75.0%), most clearly on Zhejiang (mIoU 66.2% to 73.3%). MAE and MoCo show no significant difference; the authors adopt MAE for its higher average mIoU, better average rank, and stability across both fine-tuning schemes.

• The deep learning ("DL") delineator leads on every metric and dataset in the tool benchmark. Across the internal set and LUDB, Zhejiang, and RDB, the DL delineator significantly surpasses all benchmarked tools on all three metrics, taking the best and second-best average ranks (1.3 and 1.7) versus 3.4 for ECGdeli and 3.6 for Prominence. On RDB, described as spanning a wide range of rhythms, it outperforms the strongest tool, ECGdeli, with mIoU 71.3% vs. 54.8% and averaged point-wise sensitivity 92.6% vs. 76.4%. The margin is largest on Zhejiang: mIoU 75.1% vs. 42.7% for Prominence, and averaged interval error 15.3 ms vs. 30.4 ms and averaged point-wise sensitivity 86.0% vs. 56.3% for ECGdeli.

• The advantage holds for out-of-clinic portable recordings. On mECGDB, which provides only interval-level labels, the DL delineator achieves the lowest average error, 15.0 ms (supervised) and 15.4 ms (semi-supervised), compared with 25.4 ms for the best tool, Prominence. For the PR interval the supervised model reaches 13.8 ms error versus 29.6 ms for Prominence, and the CWT variant of NeuroKit2 shows a PR error of 207.4 ms. For the QT interval, both configurations reach 19.1 ms versus 21.7 ms for Prominence.

• Each rule-based tool has a characteristic weak point. The CWT variant of NeuroKit2 is particularly weak at P onset (43.7% on LUDB, 14.7% on Zhejiang), while the DWT variant struggles at QRS onset and T offset, the two points that define the QT interval (e.g., 66.2% and 55.8% on LUDB). These match their large mECGDB errors, which are PR-driven for CWT (207.4 ms) and QT-driven for DWT (91.5 ms). CalECG, configured to output only P onset, QRS onset, QRS offset, and T offset, reaches 68.1% overall sensitivity on LUDB but drops to 7.0% overall on Zhejiang.

• Among the existing tools, ECGdeli and Prominence are the strongest. ECGdeli gives the highest QRS onset sensitivity on both LUDB and Zhejiang (97.3% and 83.2%), while Prominence is more reliable at P onset (79.2% and 65.4%); for the remaining points the leading tool depends on the dataset. In the Discussion the authors describe them as essentially indistinguishable in average rank (3.5 vs. 3.7); the Results table reports 3.4 and 3.6.

• Fine-tuning strategies differ at only a few boundaries. On LUDB, supervised fine-tuning falls short of the best tool at P offset (81.3% vs. 87.2% for ECGdeli) and T onset (77.8% vs. 79.1% for Prominence), while semi-supervised fine-tuning exceeds all tools at both (90.3% and 79.2%). On Zhejiang, both configurations lead at every point by a wide margin (e.g., T offset 85.4% vs. 45.4% for ECGdeli). The authors read these as tendencies, since a single model per configuration was used, and note the strongest overall model's weakest region is the T-wave boundaries, especially T onset.

• The deep learning model degrades least from sinus to arrhythmia. On RDB, mIoU relative degradation is 14.7% (supervised) and 16.2% (semi-supervised) versus 41.6% for ECGdeli and 48.5% for Prominence; sensitivity degradation is 16.7% versus 17.1% for ECGdeli. In the sinus group, the DL delineator leads on mIoU (76.1% and 76.6% vs. 66.3% for ECGdeli) and sensitivity, and trails Prominence slightly on interval error (19.4 ms vs. 18.2 ms). Under arrhythmia it is most accurate on every metric (mIoU 64.9% vs. 38.7%; interval error 28.5 ms vs. 38.2 ms; sensitivity 80.4% vs. 67.3%). Prominence's interval error degrades by 110.0% from sinus to arrhythmia.

Methodology in Plain English

The study has two stages.

Stage 1 — choosing the best way to train. The authors assemble seven public datasets plus one private dataset. Three are used for development: QTDB and ISP supply the labeled training data (split independently into training, validation, and test sets, then merged into a single labeled pool), while PTB-XL supplies unlabeled ECGs for semi-supervised training. For pretraining they use MIMIC-IV-ECG: from the original 800,035 diagnostic 12-lead ECGs from 161,352 subjects, filtering out corrupted recordings and those with empty or very short reports leaves 761,139 recordings from 154,094 subjects, each 10 seconds at 500 Hz. Every ECG is cropped or zero-padded to 10 seconds, resampled to 250 Hz, band-pass filtered at 0.67–40 Hz, and Z-score normalized.

They then cross two factors: whether the encoder is initialized by self-supervised pretraining on a large unlabeled dataset from a separate source or trained from scratch, and whether fine-tuning uses labeled data only (supervised) or additionally uses an unlabeled set (semi-supervised). Three pretraining objectives represent three families: MAE (generative masking and reconstruction), MoCo (unimodal contrastive), and MERL (multimodal, aligning ECG with its text report). Semi-supervised learning uses Mean Teacher, a consistency-regularization method, with a weak-strong augmentation scheme.

The architecture is held fixed across all strategies so differences come from the training strategy, not the model: a ViT-Tiny encoder with a two-layer FCN decoder, a 128-dimensional hidden layer, and dropout p = 0.1. Delineation models train for 100 epochs with batch size 16 using AdamW with weight decay 0.05 and a cosine learning rate schedule warming up to 0.001 over 10 epochs before annealing to 0.0001. Each model is trained with three random seeds; results are reported as mean plus or minus standard deviation, and the run with the median validation mIoU is used for the tool comparison. Experiments run in PyTorch 1.11 with Python 3.9 on four NVIDIA RTX 4090 GPUs.

Stage 2 — benchmarking against tools. The selected model (MAE, under both supervised and semi-supervised fine-tuning, called DL delineator (sup) and (semi))

Authors’ abstract

Electrocardiogram (ECG) delineation, the identification of waveform boundaries, is a foundational step that translates raw ECG signals into clinically interpretable measurements. Deep learning has advanced this task but remains dependent on costly expert annotations. Label-efficient strategies such as self-supervised pretraining and semi-supervised learning are expected to ease this burden, yet it remains unclear whether they yield reliable delineation and whether the deep models they produce outperform the delineation tools used in practice. We address this in two stages. First, comparing self-supervised objectives with supervised or semi-supervised fine-tuning across one internal and four external datasets, we find that pretraining helps but the objective matters, and that the value of semi-supervised fine-tuning depends on the pretraining objective. Second, we benchmark the selected deep learning model against widely used open-source (NeuroKit2, Prominence, ECGdeli) and commercial (CalECG) tools using three complementary metrics. The model ranks best on every metric and dataset, outperforming the strongest tool by a clear margin on the rhythm-diverse set (mIoU 71.3 vs. 54.8%; averaged point-wise sensitivity 92.6 vs. 76.4%), and degrades the least from sinus to arrhythmia. A rhythm-stratified and point-wise analysis further characterizes the distinctive behavior of each tool, yielding practical guidance for tool selection. These results provide systematic, multi-dataset evidence that self-supervised pretraining is effective for ECG delineation and enables a label-efficiently trained deep learning model to outperform widely used delineation tools by leveraging abundant unlabeled data. This supports adopting such models in diverse, real-world clinical settings.