Timepoint-Specific Benchmarking of Deep Learning Models for Glioblastoma Follow-Up MRI

AI summary

Overview

Research area: Medical computer vision — deep learning applied to follow-up MRI in neuro-oncology, specifically distinguishing tumor progression from treatment-related pseudoprogression in glioblastoma.

Technical level: Advanced. The paper compares transformer, recurrent, hybrid, and selective state-space architectures (including Mamba), so familiarity with modern deep learning families is helpful.

Scope: A timepoint-specific benchmark comparing eleven deep learning model families on post-radiotherapy follow-up MRI scans, evaluated separately at different follow-up points rather than pooled together.

What This Paper Is About

After radiotherapy, glioblastoma patients can show new contrast enhancement on MRI that either reflects real tumor progression (TP) or a treatment-related effect called pseudoprogression (PsP). Telling these apart is clinically difficult, particularly at early follow-up. This paper benchmarks a broad set of deep learning architectures to test whether model performance depends on when in the follow-up pathway the scan was taken, rather than assuming a single model ranking holds across all timepoints.

Key Contributions

1. A stage-specific benchmark. The authors present what they describe as the first timepoint-specific, cross-sectional benchmarking of deep learning models for glioblastoma follow-up MRI, analyzing post-radiotherapy scans independently rather than merging them into one pool.
2. Broad architecture coverage under one protocol. Eleven representative deep learning families — CNNs, LSTMs, hybrid designs, transformers, and selective state-space models — were trained through a unified, quality-control-driven pipeline with patient-level cross-validation, so comparisons are not confounded by differing training setups.
3. An accuracy-versus-efficiency characterization. The study identifies which model classes offer the best balance of performance and computational cost, and shows that this trade-off differs meaningfully across architecture families.
4. Evidence that training configuration matters. Performance proved sensitive to batch size, which the authors use to argue for standardized training protocols as a prerequisite for trustworthy benchmarking.

Main Findings

• Accuracy was comparable across stages: Across both follow-up stages, model accuracies fell in a similar range (roughly 0.70–0.74), so raw accuracy alone did not separate the timepoints.
• Discrimination improved later in the pathway: At the second follow-up, F1 and AUC increased for several models, which the authors interpret as richer separability between TP and PsP later in the care pathway.
• A Mamba+CNN hybrid offered the best accuracy-efficiency trade-off: This architecture consistently gave the strongest balance between predictive performance and computational demand.
• Transformers were competitive but costly: Transformer variants achieved competitive AUC values while requiring substantially more computation.
• Lightweight CNNs were efficient but less reliable: These models were computationally cheap but showed lower dependability in this task.
• Batch size influenced results: Performance varied with batch size, highlighting the need for standardized training protocols before drawing benchmark conclusions.
• Absolute discrimination stayed modest: The authors attribute this to the intrinsic difficulty of the TP-versus-PsP distinction and to size imbalance in the dataset. The abstract reports no further breakdown of these effects.

Methodology in Plain English

The researchers took a cohort of glioblastoma patients who had undergone radiotherapy and, instead of treating every follow-up MRI as equivalent, split the scans by how far along the follow-up schedule they were. Each timepoint was analyzed as its own problem. On top of that, they trained eleven different kinds of deep learning models — convolutional networks, recurrent (LSTM) networks, hybrids of the two, transformers, and state-space models — using one shared training and quality-control procedure, with cross-validation organized so that all scans from a given patient stayed on the same side of the train/test split. This patient-level splitting prevents the same person's images from appearing in both training and testing, which would inflate results. They then compared the models both on how well they classified cases and on how much computation they demanded.

Why This Matters

Impact on research. Most medical imaging benchmarks report a single aggregate score across a heterogeneous patient timeline. This paper argues that timepoint is a hidden variable: a model's ranking can shift depending on which follow-up stage it is evaluated on. That reframes how follow-up imaging benchmarks should be designed and suggests that pooling scans from different stages can mask real differences in model behavior. The batch-size sensitivity finding also pushes the field toward reporting and standardizing training configurations.

Real-world applications.

• Clinical decision support: Helping clinicians decide whether new enhancement after radiotherapy represents true progression or pseudoprogression, a distinction that shapes whether treatment is escalated, continued, or changed.
• Stage-aware deployment: Since discrimination differed between the first and second follow-up, a tool tuned to one stage may not transfer cleanly to another; deployment decisions can account for this.
• Compute-constrained settings: The accuracy-efficiency comparison gives practical guidance for hospitals and labs that cannot afford heavy transformer inference but still need dependable predictions.
• Avoiding unnecessary intervention: More reliable TP/PsP discrimination could reduce the harm from acting on a pseudoprogression that would have resolved on its own.

Industry relevance. Medical imaging AI developers and vendors must choose architectures under real compute, latency, and cost constraints, not just leaderboard accuracy. This benchmark offers evidence that a hybrid model can match or beat heavier alternatives on the accuracy-efficiency axis, and that batch-size choices during training affect downstream reliability — both directly relevant to productization and regulatory documentation of training procedures.

Future Directions

• Longitudinal modeling: Rather than treating each scan cross-sectionally, model the sequence of a patient's scans over time, since progression is inherently a temporal process.
• Multi-sequence MRI: Incorporate additional MRI sequences beyond what was used here, which may carry complementary signal for separating TP from PsP.
• Larger multi-center cohorts: Expand beyond a single cohort of 180 patients to test whether the stage-dependent patterns and architecture rankings generalize across institutions, scanners, and protocols.
• Standardized training and reporting: Formalize training-protocol conventions — given the observed batch-size sensitivity — so future benchmarks are comparable across studies.
• Addressing class imbalance: The abstract notes size imbalance in the dataset as a factor limiting absolute discrimination; methods that handle imbalance more effectively are an open problem.

Target Audience

Researchers in medical image analysis and computer vision who work on brain tumor imaging; neuro-oncologists and radiologists interested in the TP-versus-PsP problem; machine learning practitioners evaluating architecture choices under compute constraints; and clinical AI developers who need benchmark evidence for model selection and deployment decisions. The paper is most useful to readers comfortable with modern deep learning terminology, though the clinical framing is accessible to a medical audience.

Authors’ abstract

Differentiating true tumor progression (TP) from treatment-related pseudoprogression (PsP) in glioblastoma remains challenging, especially at early follow-up. We present the first stage-specific, cross-sectional benchmarking of deep learning models for follow-up MRI using the Burdenko GBM Progression cohort (n = 180). We analyze different post-RT scans independently to test whether architecture performance depends on time-point. Eleven representative DL families (CNNs, LSTMs, hybrids, transformers, and selective state-space models) were trained under a unified, QC-driven pipeline with patient-level cross-validation. Across both stages, accuracies were comparable (~0.70-0.74), but discrimination improved at the second follow-up, with F1 and AUC increasing for several models, indicating richer separability later in the care pathway. A Mamba+CNN hybrid consistently offered the best accuracy-efficiency trade-off, while transformer variants delivered competitive AUCs at substantially higher computational cost and lightweight CNNs were efficient but less reliable. Performance also showed sensitivity to batch size, underscoring the need for standardized training protocols. Notably, absolute discrimination remained modest overall, reflecting the intrinsic difficulty of TP vs. PsP and the dataset's size imbalance. These results establish a stage-aware benchmark and motivate future work incorporating longitudinal modeling, multi-sequence MRI, and larger multi-center cohorts.