Rare Event Early Detection: A Dataset of Sepsis Onset for Critically Ill Trauma Patients

AI summary

Overview

Research area: Machine learning for healthcare, specifically rare event early detection using electronic health records (ICU time-series data).

Technical level: Intermediate. Readers will benefit from familiarity with binary classification, class imbalance, autoencoders and recurrent/convolutional sequence models, but the clinical framing is explained from scratch.

Scope: The paper introduces a publicly released, reproducible dataset of post-traumatic sepsis onset derived from MIMIC-III, framed as a daily rare-event early detection task, and reports a benchmark of existing and extended ML methods on it.

What This Paper Is About

Existing public sepsis datasets treat all ICU patients as one uniform group and label sepsis using the general Sepsis-3 definition, which misses the fact that critically ill trauma patients already show injury-related inflammation and organ dysfunction that overlap with sepsis symptoms. The authors build and release a standardized trauma cohort with sepsis onset labels specifically designed for post-trauma patients, extracted and relabeled from MIMIC-III. They then reframe early detection as a day-by-day rare event problem aligned with ICU morning rounds and benchmark existing ML techniques on it.

Key Contributions

1. First emphasis on the distinction between general ICU sepsis and post-traumatic sepsis, with a standardized trauma cohort and targeted post-traumatic sepsis onset labels built to support early detection.
2. A clinically aligned detection framing: rather than labeling an entire visit as septic (which occurs between 13.6% and 39.3% in the ICU), each day of a visit is examined for the first possible sepsis onset, matching ICU daily evaluation and treatment planning during morning rounds. This creates a severe class imbalance, i.e., a rare event problem.
3. A public, reproducible resource: a standardized post-trauma sepsis onset dataset extracted, relabeled and validated from MIMIC-III, supported by documented code at https://github.com/ML4UWHealth/SepsisOnset_TraumaCohort.git.
4. A general benchmark established through comprehensive experiments extending existing ML techniques, showing that further advances are needed on this new dataset.

Main Findings

• Cohort size and composition: From MIMIC-III (61,532 admissions from 2001 to 2012 across six ICUs, more than 40,000 patients), the refined critically ill trauma cohort comprises 1,570 admissions. Median age was 59 (IQR 43–75); 1,032 (66%) male and 538 (34%) female. Median Injury Severity Score was 16 (4–20); median Charlson Comorbidity Index was 12 (0–12); median lactate was 3 (2–5) mmol/L. Median sum of ICU days was 10 (6–18), hospital admission days 16 (10–24), and mechanical ventilation days 7 (4–12). Discharge locations included rehab/distinct 588 (37%), dead 341 (22%), home 289 (18%), SNF 279 (18%), and other 73 (5%).
• Sepsis labels: Among the 1,570 trauma admissions, 729 had potential infections and 535 patients had sepsis. The peak for culture orders, indicating sepsis onset, occurs on day 5, consistent with clinical experience.
• Dataset instances: The "S dataset" (missing values imputed) contains 440 positive and 8,319 negative instances — 8,759 cases across 1,522 unique patients — an imbalance ratio of 0.050. The "N dataset" (raw data preserved) contains 455 positive and 8,522 negative instances — 8,977 cases across 1,535 unique patients — an imbalance ratio of 0.051.
• Physiological signal is visible early: Visualization of average trends in the four days before sepsis onset shows noticeable changes, particularly in temperature and heart rate at x = -1, suggesting delta values between consecutive nights may help detection.
• General ICU Sepsis-3 labels transfer poorly: Training on publicly available Sepsis-3 labels from the general ICU cohort and evaluating on the trauma cohort with post-trauma labels gave a degenerate result. Sepsis3-NoPre achieved sensitivity 1.000000, specificity 0.000000, precision 0.058488, F1 0.110512, PR_auc 0.029244, ROC_auc 0.499242. Sepsis3-GPre reached sensitivity 0.573171, specificity 0.400000, precision 0.056019, F1 0.102063, PR_auc 0.063371, ROC_auc 0.497570.
• Trauma-specific labels help: Models fine-tuned on the much smaller set of reliable post-traumatic sepsis labels performed substantially better than those using the large amount of general ICU sepsis labels. XGBoost* reached sensitivity 0.15 (0.04), specificity 0.85 (0.01), F1 0.07 (0.02), ROC_auc 0.53 (0.03); LightGBM* reached sensitivity 0.38 (0.09), specificity 0.68 (0.07), F1 0.10 (0.01), ROC_auc 0.55 (0.02).
• Pre-training balances performance: Without pre-training, baselines were strongly biased toward one class. GRU-TCNN with no pre-training reached sensitivity 0.28 (0.09), specificity 0.76 (0.07), F1 0.11 (0.02), ROC_auc 0.55 (0.05). With trauma-cohort pre-training (TPre), sensitivity rose to 0.59 (0.16) and specificity to 0.49 (0.05), F1 0.12 (0.03), ROC_auc 0.58 (0.04). With general ICU pre-training (GPre), sensitivity was 0.62 (0.05), specificity 0.54 (0.04), precision 0.08 (0.01), F1 0.14 (0.02), PR_auc 0.08 (0.01), ROC_auc 0.60 (0.03).
• Pre-training on the larger general ICU population was more consistent than pre-training on the trauma cohort, and masking and reconstruction-based augmentation provided better performance by covering more data variance. The mask-based augmentation with general ICU pre-training and frozen encoders gave sensitivity 0.61 (0.04), specificity 0.53 (0.06), F1 0.13 (0.01), ROC_auc 0.59 (0.03).
• Architecture ablation: With general ICU pre-training and no extra augmentation, a TCNN-only backbone gave sensitivity 0.69 (0.08), specificity 0.46 (0.08), F1 0.13 (0.02), PR_auc 0.08 (0.02), ROC_auc 0.60 (0.03); a single encoder gave sensitivity 0.63 (0.10), specificity 0.51 (0.08), ROC_auc 0.59 (0.02); a GRU-only model gave sensitivity 0.24 (0.19), specificity 0.82 (0.16), precision 0.10 (0.06), F1 0.10 (0.04), ROC_auc 0.56 (0.05). The contribution of delta values between two nights was subtle after general ICU pre-training, while combining TCNN and GRU as the backbone was important for reducing bias between classes.
• VAE-based pre-training was explored but showed no consistent advantage over the Masked Autoencoder in representation quality or downstream detection performance.
• Overall performance remains low, with F1 scores around 0.06 to 0.14 and ROC_auc around 0.52 to 0.60 across settings, indicating substantial room for future advances.

Methodology in Plain English

The authors started from MIMIC-III, an open-access anonymized ICU database. They first narrowed it down to a trauma cohort by keeping hospital admissions with at least one ICU stay and chart data, selecting trauma-related ICD-9 E-codes while excluding poisoning-related codes, restricting to adults aged 18 to 89 (patients older than 89 are recorded as 300 years old in MIMIC-III for privacy), requiring an admission of 48 hours or more, and requiring at least three days of mechanical ventilation. They used hospital admission ID (HADM_ID) as the instance identifier so the whole hospital stay is tracked, rather than individual ICU stays.

For sepsis labeling, they adapted a trauma-specific definition derived from CDC adult sepsis surveillance criteria and Sepsis-3. They preprocessed three data sources: blood cultures (kept only at or after 72 hospital hours, and only blood cultures rather than all body fluid cultures, to reduce surveillance-driven false signals), antibiotics (a curated drug list, restricted to intravenous antibiotics plus two oral agents, vancomycin and linezolid, excluding prophylactic and first-day antibiotics, requiring at least four consecutive days of treatment or treatment until death or discharge), and a modified SOFA score that omits urine output and the Glasgow Coma Scale because traumatic brain injury confounds GCS. Sepsis onset is the earliest culture timestamp where both a suspicion of infection (culture ordered within a 5-day window of antibiotic initiation) and organ failure (at least a 2-point rise in modified SOFA within a 7-day window spanning 3 days before to 3 days after the culture) are met.

For detection, they use nighttime vitals (6:00 p.m. to 6:59 a.m.) to predict sepsis within the next 24 hours, mirroring morning ICU rounds: detections are produced before rounds so staff can act on them that day. Seven vital signs are used — heart rate, systolic blood pressure, diastolic blood pressure, mean blood pressure, respiratory rate, temperature, and SpO2 — aggregated into hourly records, giving inputs of shape (T, F) with T = 13 hourly timestamps and F = 7 features. Only nights from day 2 to day 14 after admission are kept. Missing values are filled forward then backward within the night window, and instances still missing values are dropped ("S dataset"); a raw version is also released ("N dataset"). A night is labeled positive only if sepsis onset falls in the 24 hours after 6:59 a.m. of that night; earlier nights for the same patient are negative and later ones are discarded.

For modeling, they propose a two-stage pipeline: (1) reconstruction-based representation learning with a Masked Autoencoder, pre-training either on the trauma cohort itself (TPre, with minority-class oversampling plus masking) or on a general ICU population (GPre); and (2) classification on an oversampled balanced dataset using either mask-based augmentation or reconstruction-based augmentation. Baselines include Sepsis3-NoPre, Sepsis3-GPre, XGBoost, LightGBM, and GRU-TCNN variants, with 5-fold cross-validation on the trauma-only dataset except for the two large general-ICU settings.

Why This Matters

Impact on research: The paper argues that mixing trauma patients into general ICU sepsis datasets produces labels that do not reflect post-trauma physiology, and that reproducibility in prior trauma-specific sepsis work was limited because datasets were private or code was unavailable. It supplies a standardized cohort, trauma-specific labels, both an imputed and a raw version of the data, and a documented reproducible code base, giving the community a shared rare-event benchmark that extends beyond sepsis to rare event detection in general.

Real-world applications:

• ICU early warning: generating a pre-rounds risk signal so clinicians can incorporate sepsis risk into daily care planning for trauma patients.
• Post-traumatic sepsis surveillance and quality reporting in trauma and surgical ICUs, using a definition adapted from CDC surveillance criteria.
• Training and evaluating new rare-event ML methods where the minority class is both a small fraction and limited in absolute count.
• Cross-database generalization studies: the same pipeline is proposed for extension to other databases such as MIMIC-IV and eICU.

Industry relevance: Hospital systems, EHR vendors, and clinical AI developers building sepsis alerting tools currently rely on general-population models; this dataset provides a specialized trauma cohort for validating such tools, and the demonstration that general ICU Sepsis-3 labels transfer poorly to trauma patients is directly relevant to alert design and false-positive burden. The public code base also supports regulatory-style reproducibility expectations for clinical ML.

Future Directions

• Extend the pipeline to other databases, such as MIMIC-IV and eICU, to improve generalizability of the dataset and labels.
• Improve rare event detection methods, since benchmark F1 scores remain around 0.06 to 0.14 and ROC_auc around 0.52 to 0.60, leaving clear room for advancement.
• Enrich the feature set beyond the seven nighttime vital signs, for example adding laboratory results, with the repository to be updated and maintained accordingly.
• Close the gap between pre-training sources, given that general ICU pre-training was more consistent than trauma-cohort pre-training, and that VAE-based pre-training showed no consistent advantage over the Masked Autoencoder.

Target Audience

ML researchers working on rare event detection, class imbalance and clinical time-series modeling; clinical informatics and critical care researchers interested in post-traumatic sepsis phenotypes and surveillance definitions; and industry or hospital teams building or validating sepsis early warning systems, particularly those targeting trauma and surgical ICU populations.

Authors’ abstract

Sepsis is a major public health concern due to its high morbidity, mortality, and cost. Its clinical outcome can be substantially improved through early detection and timely intervention. By leveraging publicly available datasets, machine learning (ML) has driven advances in both research and clinical practice. However, existing public datasets consider ICU patients (Intensive Care Unit) as a uniform group and neglect the potential challenges presented by critically ill trauma patients in whom injury-related inflammation and organ dysfunction can overlap with the clinical features of sepsis. We propose that a targeted identification of post-traumatic sepsis is necessary in order to develop methods for early detection. Therefore, we introduce a publicly available standardized post-trauma sepsis onset dataset extracted, relabeled using standardized post-trauma clinical facts, and validated from MIMIC-III. Furthermore, we frame early detection of post-trauma sepsis onset according to clinical workflow in ICUs in a daily basis resulting in a new rare event detection problem. We then establish a general benchmark through comprehensive experiments, which shows the necessity of further advancements using this new dataset. The data code is available at https://github.com/ML4UWHealth/SepsisOnset_TraumaCohort.git.