Hierarchical Spatio-Temporal Transformer for Coherent Emergency Department Forecasting

AI summary

Overview

Research area: Hierarchical time-series forecasting applied to healthcare demand prediction, combining attention-based deep learning (Temporal Fusion Transformers, spatio-temporal Transformer encoder–decoders) with multi-level forecast reconciliation.

Technical level: Intermediate. The paper uses well-established Transformer building blocks and standard forecasting metrics, but assumes familiarity with attention mechanisms, encoder–decoder architectures, and hierarchical time-series concepts such as coherence and reconciliation.

Scope: The paper proposes HierSTT, a single end-to-end hierarchical Transformer that jointly forecasts Emergency Department demand at hospital, regional, and national levels in Portugal while penalizing cross-level incoherence, and introduces a supporting nationwide dataset.

What This Paper Is About

Emergency Departments face unpredictable demand, seasonal surges, and non-urgent visits, and different decision-makers need forecasts at different scales: hospitals need local estimates for staffing and beds, regions need to coordinate units, and national authorities need system-wide projections. Most existing methods forecast at a single level in isolation, so hospital-level predictions may not add up consistently to regional or national numbers, producing incoherent plans. HierSTT addresses this by jointly predicting all three levels in one model, guided by higher-level forecasts and a loss term that penalizes inconsistencies between levels.

Key Contributions

1. A hierarchical Transformer framework (HierSTT) that simultaneously forecasts ED demand at hospital, regional, and national levels in a single end-to-end model, using top-down conditioning in which national forecasts inform regional decoding and regional forecasts inform hospital decoding.
2. A coherence-aware training objective that adds a soft penalty term to the loss, comparing predictions across levels rather than comparing aggregated predictions to ground truth, so that lower-level forecasts are encouraged to aggregate consistently with higher-level ones without being hard-constrained.
3. A new nationwide Portuguese ED dataset covering 81 hospitals grouped into 5 Regional Health Administrations (RHAs) plus national aggregates, with heterogeneous, level-specific covariates.
4. An evaluation combining accuracy and coherence, reporting MAE, RMSE, WAPE and a Hierarchical Aggregation Error (HAgE) metric measured both against higher-level predictions and against higher-level ground truth.

Main Findings

• Average WAPE reduction: HierSTT reduces average WAPE by 32% relative to the best non-hierarchical deep learning baseline (abstract), and by over 30% relative to the strongest competitors, named as N-BEATS, Naïve, and FA-Transformer (results and conclusion).
• Outperforms reconciliation methods: HierSTT outperforms all classical hierarchical reconciliation methods tested (Bottom-Up, Top-Down, Middle-Out applied to statistical forecasts via the Nixtla framework).
• Hospital level: HierSTT achieves the lowest MAE, improving by approximately 8% over N-BEATS, the second-best model. It has slightly higher RMSE at this level, indicating greater sensitivity to peak deviations; N-BEATS achieves the lowest hospital-level RMSE.
• Regional level: HierSTT reduces MAE and RMSE by approximately 38% and 27% respectively over the second-best Naïve baseline.
• National level: HierSTT achieves the best RMSE and is essentially tied with the standalone TFT on MAE, indicating the top-level forecaster retains its performance inside the hierarchy.
• Coherence (prediction-side HAgE): HierSTT achieves the best prediction-side HAgE overall, improving by approximately 13% over ARIMA, the best statistical model on this measure. Naïve and reconciliation-based models achieve zero prediction-side HAgE by construction and are excluded from that comparison.
• Coherence (ground-truth HAgE): HierSTT outperforms all models including Naïve and reconciliation baselines, reducing ground-truth HAgE by approximately 18% relative to Seq2seq-LSTM, the second-best.
• Positioning across levels: HierSTT is described as the only model consistently positioned in the lower-left region (low error, low incoherence) across all three levels, whereas competitors tend to excel at one level at the expense of others; TFT is cited as an example of strong national accuracy but poor hospital-level performance.
• Stability: Across six fixed random seeds, error bars indicate HierSTT is the most stable deep learning model. The standard Transformer shows the worst prediction-side coherence among unconstrained models; ARIMA performs best among statistical models.
• Coherence weight: α = 0.3 provided the best accuracy–coherence trade-off among tested values, with detailed results reported in the supplementary material.

Methodology in Plain English

The researchers treat the Portuguese health system as a three-level hierarchy: 81 individual hospitals roll up into 5 regional health administrations, which in turn roll up into one national total. Each level is given a 42-day history of daily ED visits plus covariates, and must produce a 28-day (4-week) forecast.

Forecasting runs from the top down. A Temporal Fusion Transformer first predicts the national series, using past and future-known variables such as calendar indicators, waiting times, and mortality counts, with variable selection networks to weight which inputs matter. The national forecast is then fed into the decoder of a spatio-temporal Transformer that predicts the 5 regions; that regional forecast is fed into an equivalent module for the 81 hospitals. These regional and hospital modules alternate between temporal attention (patterns over time within one entity) and spatial attention (relationships across entities at the same time step), with cross-attention linking the decoder to encoded history.

Training optimizes a blended loss: a weighted sum of direct Smooth L1 forecasting losses at each of the three levels, plus a coherence term that measures how far the sum of lower-level predictions deviates from the corresponding higher-level predictions. Three constraints are imposed — national versus summed regional, national versus summed hospital, and each region versus the sum of its hospitals — averaged over the 5 regions. Because these comparisons are between predictions at different levels rather than against ground truth, the coherence term is not redundant with the accuracy losses.

Data handling: visits are log-transformed as log(1+y) and features are normalized with RobustScaler fitted on training data; predictions are inverse-scaled and mapped back with expm1 before coherence is computed in count space. Data before August 2021 was dropped to reduce COVID-19 distortion, leaving 713 training days, 140 validation days, and 140 test days, with no shuffling. With a one-day sliding stride, this yields 644 training samples and 71 samples each for validation and test. Feature counts are 16, 17, and 26 at the national, regional, and hospital levels respectively. Models were trained up to 200 epochs with AdamW, OneCycleLR (peak learning rate 3×10⁻⁴, weight decay 10⁻⁴), gradient clipping at 1.0, and early stopping at 50-epoch patience, on an Intel Xeon Platinum 8260 CPU (2.40 GHz) and NVIDIA RTX A6000 GPU in PyTorch.

Why This Matters

Impact on research. The paper argues that no prior work jointly addresses hierarchical coherence and heterogeneous multi-level covariates in ED forecasting. It shows that accuracy and coherence must be evaluated together — a model can look competitive on error metrics per level while producing incoherent aggregates — and it releases both a model and a dataset for a real national health system.

Real-world applications:

• Hospital staffing and bed management, using hospital-level forecasts that still reconcile with regional totals.
• Regional coordination of healthcare units, where reconciled regional demand supports resource sharing between hospitals.
• National capacity planning, where system-wide projections must be consistent with the sum of local units.
• Operational monitoring of ED pressure, including non-urgent visits and triage-category mix, which the dataset captures.

Industry relevance. Hierarchical forecasting with coherence constraints transfers directly to retail and supply-chain demand planning across item, store, and chain levels, as well as energy load forecasting across substations and grids. The paper's approach of learning coherence during training rather than reconciling forecasts afterward is relevant wherever organizations must produce plans at several organizational levels that must add up.

Future Directions

• Tuning the coherence-accuracy trade-off: the ablation in Appendix 0.B on the coherence weight α (mentioned but truncated in the provided content) raises the question of how to select α systematically rather than by search.
• Extending beyond three levels and beyond Portugal: the dataset covers 81 hospitals and 5 RHAs in one country; whether the top-down conditioning scheme scales to deeper hierarchies or other health systems is untested here.
• Handling the peak-deviation weakness: HierSTT has slightly higher hospital-level RMSE than N-BEATS, so improving sensitivity to demand peaks without sacrificing coherence is an open issue.
• Richer cross-level covariates: environmental variables such as AQI and temperature are only available at the regional level because reliable hospital-specific measurements are limited; finer-grained environmental data could sharpen hospital-level forecasts.

Target Audience

This paper is most useful for time-series forecasting researchers working on hierarchical or coherent prediction, machine learning practitioners applying Transformers to healthcare demand, and health-system analysts and policymakers who need forecasts that reconcile across hospital, regional, and national planning levels. It is also relevant to operations researchers in retail, energy, and logistics who face the same multi-level aggregation problem. Readers need comfort with deep learning architectures and forecasting metrics; the healthcare domain background is explained clearly enough for non-clinicians.

Authors’ abstract

Emergency Departments (EDs) are critical access points in healthcare systems, yet they face persistent pressure from unpredictable patient demand, seasonal surges, and non-urgent visits. Effective ED planning requires forecasts at multiple decision-making levels: hospitals need local demand estimates for staffing and bed management, regions require forecasts to coordinate healthcare units, and national authorities need system-wide projections for capacity planning. However, most existing approaches forecast ED demand independently at a single level, ignoring the hierarchy linking hospitals, regions, and national systems. This can produce incoherent predictions, where hospital-level forecasts do not aggregate consistently to regional or national demand. We propose HierSTT, a hierarchical Transformer-based framework for coherent multi-level ED forecasting. HierSTT jointly predicts hospital, regional, and national level demand in a single end-to-end model. A Temporal Fusion Transformer captures national dynamics, while spatio-temporal Transformer encoder-decoder modules model regional and hospital demand conditioned on higher-level forecasts. A coherence-aware loss penalizes cross-level inconsistencies during training. We further introduce a nationwide Portuguese ED dataset covering 81 hospitals across 5 regional health administrations, with heterogeneous covariates at each level. Experiments show that HierSTT reduces average WAPE by 32\% relative to the best non-hierarchical deep learning baseline and outperforms all classical hierarchical reconciliation methods, while producing near-coherent predictions across levels. Additional resources associated with this work are available at https://github.com/FilipaLino/HierSTT.