Enhancing Multivariate Time Series Forecasting with Global Temporal Retrieval

AI summary

Overview

Research area: Multivariate time series forecasting (MTSF), specifically explicit modeling of long-range ("global") periodic structure in deep forecasting architectures.

Technical level: Advanced. The paper assumes familiarity with forecasting backbones (MLPs, Transformers, Mamba-based models), seasonal–trend decomposition, FFT-based frequency modeling, and 2D-convolution representations of 1D series.

Scope in one sentence: The paper proposes the Global Temporal Retriever (GTR), a lightweight, plug-and-play module that gives any forecasting backbone access to periodic patterns spanning cycles longer than the input window, and validates it on six real-world datasets in both long- and short-term settings.

What This Paper Is About

Existing forecasting models read only a fixed look-back window, so periodic patterns whose cycle length exceeds that window stay invisible to them—even though those distant, global patterns can carry stronger predictive signal than adjacent local ones. Simply lengthening the input window is impractical because it invites overfitting to noise, inflates compute and memory costs, and buries the useful signal in redundancy. The paper's goal is to give a forecaster global-cycle awareness without changing its architecture or blowing up its cost, using a small retrieval module that learns an embedding of the entire cycle.

Key Contributions

1. Problem diagnosis: The authors identify a core limitation of current MTSF methods—fixed look-back windows obscure global periodic patterns whose cycles exceed the observed history.
2. The GTR module: A lightweight, plug-and-play, model-agnostic module that uses absolute temporal indexing to retrieve from an adaptive global cycle embedding and fuses local and global cues via 2D convolution, requiring no modification to the host forecaster.
3. Broad empirical validation: Across six benchmarks and both long- and short-term settings, GTR consistently improves diverse forecasting backbones and achieves overall state-of-the-art accuracy.
4. Efficiency demonstration: The technique incurs minimal parameter and runtime overhead (measured in parameters, MACs, and per-epoch training time).

Main Findings

• Long-term forecasting results: With look-back fixed at T = 96 and horizons S ∈ {96, 192, 336, 720}, GTR achieved top-2 performance in 10 of 16 prediction tasks; its "Count" row values are 5 and 5. On the Solar-Energy dataset, GTR exceeds CycleNet by 8.2% in MSE and 6.5% in MAE. On Traffic, GTR shows higher MSE than S-Mamba, SOFTS, and iTransformer—the authors attribute this to those models better capturing inter-variable relationships and traffic's spatiotemporal dependencies and temporal lag effects.

• Short-term forecasting results: With T = 96 and horizons S ∈ {12, 24, 48, 96} on the PEMS datasets, GTR achieved top-2 performance in all 8 prediction tasks. Relative to iTransformer, GTR reduces MSE by 18.7% and MAE by 12.1% on average across all datasets and horizons; relative to S-Mamba, it achieves 15.7% MSE and 9.6% MAE reductions, with the largest gains on PEMS03 (28.7% MSE and 20.6% MAE reduction).

• Backbone improvement (ablation): GTR reduces the MLP backbone's average MSE by 14.4% on Electricity, 55.1% on PEMS03, and 73.5% on PEMS04. On PEMS04, GTR reduces MSE by 91.9% for DLinear and 56.2% for PatchTST. Even when paired with iTransformer (a model built to capture inter-variable relationships), GTR cut MSE by 62.2% on PEMS03 and 37.9% on PEMS04.

• Robustness to short inputs: Across varying look-back lengths (horizons fixed at 336 for Electricity and Solar-Energy, and 96 for PEMS03 and PEMS04), baseline models show exponential error growth as the window shortens, while GTR shows only marginal degradation—supporting the claim that it recovers global periodic information inaccessible to baselines.

• Efficiency: On Electricity with T = 96 and S = 720, the GTR technique alone uses 40.1K parameters and 4.50M MACs. Combined with the MLP backbone it uses 0.98M parameters and 306.91M MACs, with 22.3 s training time per epoch—19.0% of iTransformer's parameter count, and slower per epoch than only DLinear (18.1 s) among the compared models.

• Correlation alignment: A Pearson-correlation visualization on four datasets indicates that after GTR, learned multivariate correlations align more closely with the correlation structure computed across the entire dataset.

Methodology in Plain English

The key idea. Instead of looking further back in raw data, GTR learns a compact parameter matrix Q of shape L × N, where L is the global cycle length and N is the number of variables. This matrix is initialized to zero and trained to represent the periodic pattern of the whole cycle.

Step 1 — Locate the input within the cycle. A cycle index vector maps each time step in the input to its absolute position within the global cycle, using the sequence's absolute start time and the modulo operation.

Step 2 — Retrieve. The corresponding rows of Q are pulled out for each input position and passed through a linear layer to produce a "global query" sequence q of the same length as the input.

Step 3 — Fuse local and global. The input sequence and the retrieved global query are stacked into a 2 × T matrix, and a 2D convolution with a kernel sized (2, 1 + 2⌊P/2⌋)—where P is the dominant high-frequency period length such as daily patterns in hourly data—extracts interactions across both scales.

Step 4 — Residual injection. The convolution output is added back to the original input through a dropout-regularized residual connection, preserving the input's dimensionality so any backbone can consume it unchanged.

Backbone and training. The enhanced representation feeds a simple MLP backbone: a linear input projection to a hidden dimension D = 512, two linear layers with GeLU activations and a residual connection, and a linear output projection after dropout. Reversible Instance Normalization (RevIN) is applied to mitigate distribution shift. Experiments use PyTorch on a single NVIDIA RTX 3090 24GB GPU with the Adam optimizer, learning rates selected from {1e-3, 3e-3, 5e-4}.

Cost. Total complexity is O(NT² + Nd² + NTd + NSd), linear in the number of variables N and the horizon S; when T is much smaller than d, the O(Nd²) term dominates.

Why This Matters

Research impact. The paper reframes a widely assumed constraint—that a forecaster can only use its look-back window—as a design choice rather than a law. By showing that a small learned cycle embedding plus retrieval can substitute for a long input window, it offers an alternative to the two dominant paradigms it surveys (seasonal–trend decomposition and frequency-domain modeling), both of which remain bounded by the observed window or by stationary-cycle assumptions. It also positions itself against retrieval-augmented forecasting, which relies on similarity search quality and does not provide a compact, time-aligned representation of very long cycles.

Real-world applications (all domains named in the paper):

• Energy grid management and electricity load forecasting
• Climate and weather modeling
• Macroeconomic planning
• Traffic flow management and road-occupancy forecasting

Industry relevance. The efficiency profile matters for deployment: 0.98M parameters and 306.91M MACs when combined with the MLP backbone, 22.3 s per epoch, and the ability to bolt onto an existing forecaster without architectural changes. The paper explicitly targets resource-constrained settings and cases where historical data is severely limited.

Future Directions

• Adaptive cycle modeling: GTR assumes a single fixed cycle length for the entire sequence, which is suboptimal for time-varying periodicity (the authors cite physiological signals) and for multivariate data where different channels have heterogeneous periods.
• Handling cross-channel periodic heterogeneity: The current design enforces one shared cycle length across all channels; channel-specific modeling or preprocessing is suggested but acknowledged to add architectural complexity or data-handling overhead.
• Extreme long-horizon periodicity: Learning yearly or multi-year patterns requires decades of continuous high-quality data, which is rarely available. The 2D convolution also scales as O(NTP) with cycle length P, and kernel parameter count grows linearly with P, raising memory cost and overfitting risk.
• Scalability with long input sequences: GTR keeps the input length and uses a T × T linear projection, giving O(NT²) complexity that becomes inefficient for long sequences. The authors suggest hierarchical or memory-augmented mechanisms as possible remedies.

Target Audience

Researchers and graduate students working on time series forecasting who want to understand how explicit global-periodicity modeling can be layered onto existing backbones; practitioners deploying forecasting systems in energy, weather, traffic, or economics who face short input windows and tight compute budgets; and ablation-minded readers who want a concrete case study of a plug-and-play module evaluated across MLP, Transformer, and linear baselines. Readers without grounding in forecasting architectures and 2D-convolution representations will find the method section demanding.

Authors’ abstract

Multivariate time series forecasting (MTSF) plays a vital role in numerous real-world applications, yet existing models remain constrained by their reliance on a limited historical context. This limitation prevents them from effectively capturing global periodic patterns that often span cycles significantly longer than the input horizon - despite such patterns carrying strong predictive signals. Naive solutions, such as extending the historical window, lead to severe drawbacks, including overfitting, prohibitive computational costs, and redundant information processing. To address these challenges, we introduce the Global Temporal Retriever (GTR), a lightweight and plug-and-play module designed to extend any forecasting model's temporal awareness beyond the immediate historical context. GTR maintains an adaptive global temporal embedding of the entire cycle and dynamically retrieves and aligns relevant global segments with the input sequence. By jointly modeling local and global dependencies through a 2D convolution and residual fusion, GTR effectively bridges short-term observations with long-term periodicity without altering the host model architecture. Extensive experiments on six real-world datasets demonstrate that GTR consistently delivers state-of-the-art performance across both short-term and long-term forecasting scenarios, while incurring minimal parameter and computational overhead. These results highlight GTR as an efficient and general solution for enhancing global periodicity modeling in MTSF tasks. Code is available at this repository: https://github.com/macovaseas/GTR.