Automated Machine Learning in Radiomics: A Comparative Evaluation of Performance, Efficiency and Accessibility

AI summary

Overview

Research area: Radiomics and automated machine learning (AutoML) applied to medical imaging-based classification.

Technical level: Intermediate. The concepts are accessible to anyone with basic machine learning familiarity, but the paper compares named AutoML frameworks across multiple datasets and evaluates AUC, runtime, and software accessibility, which assumes some grounding in model evaluation and tooling.

Scope (one sentence): A comparative evaluation of six general-purpose and five radiomics-specific AutoML frameworks across ten heterogeneous radiomics datasets, assessing performance, efficiency, and accessibility to identify where current tooling falls short for radiomics.

What This Paper Is About

AutoML promises to let researchers without programming expertise build predictive and prognostic models from medical imaging data, removing a major technical barrier in radiomics. The open question is whether existing AutoML frameworks actually work well for radiomics, which has its own peculiarities (imaging modalities, feature reproducibility, harmonization, survival endpoints). This study benchmarks general-purpose against radiomics-specific frameworks to measure how well each handles those radiomics-specific demands, and to point out what still needs to be built.

Key Contributions

1. A head-to-head benchmark of AutoML frameworks for radiomics. Six general-purpose and five radiomics-specific frameworks were evaluated under a common protocol.
2. A deliberately heterogeneous evaluation set. Ten public and private radiomics datasets were used, spanning CT and MRI, varied dataset sizes, different anatomical sites, and different endpoints.
3. A multi-dimensional evaluation beyond accuracy. The comparison combined quantitative metrics (AUC and runtime) with qualitative dimensions: software status, accessibility, and interpretability.
4. A gap analysis for the field. The study documents concrete shortcomings in current AutoML tooling, specifically the lack of accessible survival analysis support and the weak integration of feature reproducibility and harmonization.

Main Findings

• A radiomics-specific, no-code tool led on accuracy. Simplatab achieved the highest average test AUC at 81.81%, with a moderate runtime of roughly one hour.
• A general-purpose framework was the fastest. LightAutoML produced a competitive mean AUC of 78.74% in about six minutes, the quickest execution reported.
• Most radiomics-specific frameworks could not be benchmarked at all. They were excluded from the performance analysis because of obsolescence, extensive programming requirements, or computational inefficiency — a notable finding in itself about the maturity of the radiomics tooling ecosystem.
• General-purpose frameworks were easier to adopt. They demonstrated higher accessibility and ease of implementation than their radiomics-specific counterparts.
• Simplatab offered the best overall balance. It combined strong performance, reasonable runtime, and a no-code interface for radiomics classification problems.
• Two structural gaps remain unaddressed. Current AutoML frameworks lack accessible survival analysis support and do not meaningfully integrate feature reproducibility and harmonization.
• Details not in the abstract. The abstract does not report per-dataset results, statistical significance testing, the specific cross-validation scheme, hardware used, or how the qualitative accessibility and interpretability assessments were scored. Whether any evaluated framework was developed by the authors is also not stated.

Methodology in Plain English

The researchers assembled a set of ten radiomics datasets drawn from both public and private sources, chosen to vary widely: different imaging modalities (CT and MRI), different numbers of patients, different body regions, and different clinical targets. They then selected eleven AutoML frameworks — six designed for general machine learning and five built specifically for radiomics — and ran each one on these datasets using predefined settings rather than per-dataset tuning, so the comparison stayed fair. Every framework was evaluated with the same standardized cross-validation procedure. Performance was measured primarily with AUC, and practical cost was measured with runtime. Alongside these quantitative measures, the team made qualitative judgments about each tool's software status, how easy it was to access and use, and how interpretable its outputs were. Notably, several radiomics-specific frameworks could not be run through the full comparison and were excluded, which is itself part of the paper's reported result. The abstract does not describe the number of cross-validation folds, the computing environment, or the exact parameter values used.

Why This Matters

Impact on research. The paper shifts the conversation from "can AutoML work in radiomics?" to "which AutoML actually works, and what is still missing?" It gives researchers an evidence-based starting point for tool selection and, more importantly, identifies specific engineering gaps — survival analysis and feature harmonization — that the field must close before AutoML can be considered a complete solution for radiomics. The exclusion of most radiomics-specific frameworks due to obsolescence is a warning about tool longevity in academic software.

Real-world applications:

• Clinical research groups without dedicated machine learning engineers can use no-code AutoML to build classification models from their CT or MRI cohorts.
• Radiomics studies aiming to distinguish patient groups (for example, benign versus malignant findings) can compare a fast general-purpose framework against a slower but more accurate radiomics-specific option.
• Teaching and methods training, where a no-code interface lowers the entry barrier for clinicians learning predictive modeling.
• Institutional tool procurement, where the accessibility and software-status findings inform which platform is sustainable to adopt.

Industry relevance. Medical imaging AI vendors and platform developers get a clear picture of the competitive landscape and of unmet customer needs, particularly around survival modeling and image-feature harmonization, which are common demands in real clinical data. The runtime figures also matter commercially, since turnaround time affects how AutoML can fit into research and clinical workflows.

Future Directions

1. Build accessible survival analysis into AutoML. The abstract names this as a significant remaining gap, and it is a core need for prognostic radiomics work.
2. Integrate feature reproducibility and harmonization. AutoML frameworks currently handle model building but not the upstream problem of ensuring that radiomic features are stable and comparable across scanners, protocols, and sites.
3. Modernize or replace the radiomics-specific frameworks that fell out of use. Obsolescence and heavy programming requirements eliminated most of them from the benchmark, so there is a clear opening for maintained, usable radiomics-native tools.
4. Adapt AutoML more broadly to radiomics-specific challenges. The abstract's recommendation is general: future work should tailor AutoML solutions to the domain rather than expecting general-purpose frameworks to suffice. Open questions include how these frameworks perform on larger multi-center cohorts and how their qualitative accessibility and interpretability should be measured rigorously — the abstract does not specify these details.

Target Audience

Radiomics researchers and imaging scientists who build or consume predictive models; clinicians and clinical researchers without programming backgrounds who want to know whether no-code AutoML is a realistic option; machine learning practitioners entering the medical imaging domain; and developers or vendors building AutoML platforms, who will find the enumerated gaps the most actionable part of the paper. Readers seeking detailed numerical comparisons should note that only the top-line AUC and runtime figures appear in the abstract, with fuller results presumably in the body of the paper.

Authors’ abstract

Automated machine learning (AutoML) frameworks can lower technical barriers for predictive and prognostic model development in radiomics by enabling researchers without programming expertise to build models. However, their effectiveness in addressing radiomics-specific challenges remains unclear. This study evaluates the performance, efficiency, and accessibility of general-purpose and radiomics-specific AutoML frameworks on diverse radiomics classification tasks, thereby highlighting development needs for radiomics. Ten public/private radiomics datasets with varied imaging modalities (CT/MRI), sizes, anatomies and endpoints were used. Six general-purpose and five radiomics-specific frameworks were tested with predefined parameters using standardized cross-validation. Evaluation metrics included AUC, runtime, together with qualitative aspects related to software status, accessibility, and interpretability. Simplatab, a radiomics-specific tool with a no-code interface, achieved the highest average test AUC (81.81%) with a moderate runtime (~1 hour). LightAutoML, a general-purpose framework, showed the fastest execution with competitive performance (78.74% mean AUC in six minutes). Most radiomics-specific frameworks were excluded from the performance analysis due to obsolescence, extensive programming requirements, or computational inefficiency. Conversely, general-purpose frameworks demonstrated higher accessibility and ease of implementation. Simplatab provides an effective balance of performance, efficiency, and accessibility for radiomics classification problems. However, significant gaps remain, including the lack of accessible survival analysis support and the limited integration of feature reproducibility and harmonization within current AutoML frameworks. Future research should focus on adapting AutoML solutions to better address these radiomics-specific challenges.