AliO: Output Alignment Matters in Long-Term Time Series Forecasing

AI summary

Overview

Research area: Long-term time series forecasting (LTSF), specifically the reliability/consistency of deep forecasting model outputs rather than raw accuracy.

Technical level: Intermediate. The core idea is a training loss addition, but it involves Fourier (frequency domain) alignment, a new evaluation metric, and stop-gradient mechanics.

Scope: The paper identifies and names the "output alignment" problem in LTSF, proposes a new metric (TAM) to measure it, and proposes a method (AliO) that improves alignment while maintaining or improving forecasting error.

What This Paper Is About

Long-term time series forecasting models are trained to minimize the distance between their predictions and the ground truth, but nothing forces them to produce the same prediction for the same future timestamp when the input window is shifted slightly. The paper shows that state-of-the-art models therefore output fluctuating predictions for overlapping timestamps across lagged inputs, which undermines reliability. The goal is to make those overlapping predictions consistent in both the time and frequency domains, without sacrificing (and ideally while improving) standard forecasting accuracy.

Key Contributions

1. Identification of the output alignment problem. The authors state that "none of existing works has explicitly recognized or attempted to address this inconsistency in LTSF tasks," making this the first study they know of to address consistency improvement in LTSF.

2. A new metric, TAM (Time Alignment Metric). TAM quantifies the alignment between overlapping prediction outputs by averaging the MAE between all pairs of overlapping output sequences generated from lagged inputs. The paper states this is the first metric designed to measure output alignment, contrasting it with regression metrics (MSE, MAE, DTW) that only measure distance to ground truth.

3. AliO, a loss-based alignment method. AliO adds a time domain alignment loss and a frequency domain alignment loss to the standard regression loss, using a technique called regression pulling (RegPull) with a stop-gradient operation so the regression loss direction is preserved.

4. Empirical validation across models and datasets. AliO is applied to seven model families (CycleNet, GPT4TS, iTransformer, PatchTST, TimesNet, DLinear, Autoformer) on ETTh1, ETTh2, ETTm1, ETTm2, Electricity (ECL), Traffic, Weather, and ILI. Code is released at https://github.com/eai-lab/AliO.

Main Findings

• Alignment improves dramatically. In Table 1 (seven datasets excluding ILI), AliO improves TAM by up to 70.5% for CycleNet, 45.8% for GPT4TS, 45.8% for iTransformer, 36.8% for PatchTST, 45.6% for TimesNet, 64.0% for DLinear, and 39.0% for Autoformer. The abstract cites an overall TAM improvement of up to 58.2%.

• Accuracy is maintained or improved. Table 1 reports MSE improvements of up to 11.5%. The abstract states forecasting performance improves by up to 27.5%, while the introduction states up to 17.4% measured in MSE; the paper does not reconcile these two figures in the provided content.

• The largest gains appear where alignment was worst. On the ILI dataset, which the authors describe as exhibiting insufficient initial output alignment, AliO achieves 17.4% MSE improvement and 58.1% TAM improvement. ILI results (Table 2): GPT4TS 1.898 MSE / 0.117 TAM to 1.755 / 0.098; PatchTST 1.813 / 0.110 to 1.497 / 0.046; TimesNet 2.176 / 0.322 to 1.963 / 0.258; DLinear 2.247 / 0.152 to 2.187 / 0.119.

• Initial misalignment correlates with MSE gains. Figure 4 reports a correlation coefficient of 0.33 between initial TAM and MSE improvement percentage, meaning that poorly aligned baselines tend to gain more from AliO.

• Both domains matter. Frequency domain alignment is less influential than the time domain coefficient, but it matters when time-domain alignment alone cannot improve performance; the paper cites example values of 0.28 and 0.50 for MSE and TAM respectively, and states that both phase and amplitude contribute to the gains.

• Frequency-domain MSE provably aligns phase and amplitude. Theorem 4.1 shows that driving the MSE between two frequency-domain prediction vectors to zero also drives their phase difference to zero; Theorem 4.2 shows that once phases are aligned, minimizing the same MSE reduces the amplitude difference.

• Sensitivity to hyperparameters. Increasing the time domain coefficient λ_T from 1.0 to 5.0 consistently improves TAM and generally improves normalized MSE. For the number of overlapping predictions N and lag l, the paper states that increasing N improves TAM while increasing l degrades performance, with no distinct tendencies observed for MSE (the detailed discussion of l is cut off in the provided content).

Methodology in Plain English

The starting point is straightforward: instead of training a model only to match ground truth, the authors also feed it several lagged input windows — inputs that are shifted by a small number of timestamps — and compare the parts of the resulting predictions that cover the same future timestamps.

For each pair of overlapping predictions, the method figures out which of the two is further from the ground truth at each timestamp. The one that is further away is "pulled" toward the other one, and a stop-gradient is applied to the closer one so that it acts as a fixed target rather than being dragged off course. This is Regression Pulling (RegPull). The effect is that the alignment loss pushes predictions in the same direction the regression loss already pushes them, which is why accuracy does not degrade.

The same comparison is repeated after applying an FFT, so the overlapping predictions are also compared in the frequency domain using MSE. Because minimizing complex-domain MSE reduces both phase and amplitude differences (Theorems 4.1 and 4.2), this complements the time-domain alignment.

The two alignment losses are weighted by λ_T and λ_F and added to the ordinary regression loss, so AliO is simply a training loss modification requiring no changes to model architecture. In the experiments, N = 2 and l = 1 were used, λ_T was searched over {1.0, 2.0, 5.0} and λ_F over {0.0, 0.5, 1.0, 2.0}, with MSE as the regression loss and the best results reported.

Why This Matters

Research impact: The paper reframes part of the LTSF evaluation problem. It argues that MSE, MAE, MAPE, RMSE and DTW all measure distance to ground truth and therefore cannot capture consistency between repeated predictions, and it introduces TAM to fill that gap alongside a training method that targets it.

Real-world applications:

• Electricity demand planning: The motivating example is a utility planning budget allocation where two forecasts of the same months disagree, forcing costly rescheduling.
• Weather forecasting: The paper cites studies that consistent forecasts increase user trust while inconsistent forecasts cause distrust.
• Health data / illness forecasting: The ILI dataset experiments target forecasting of illness-related series where unstable predictions would hamper planning.
• Traffic forecasting: Traffic is one of the benchmark datasets evaluated.

Industry relevance: Rolling forecasting is used operationally for resource allocation, and the paper argues that inconsistency forces replanning, extra resource investment, and sunk costs (citing work on rolling forecasting, sunk costs, and opportunity costs). A loss function that can be dropped into an existing model without architectural change lowers the barrier to adoption for practitioners.

Future Directions

• Extending beyond N = 2 and l = 1. The main experiments fix the number of overlapping predictions at 2 and the lag at 1; the paper's own analysis indicates behavior changes with larger N and l, so the practical setting of these parameters is left for further characterization.
• Explaining the gap between reported improvement figures. The abstract's 27.5% forecasting improvement and the introduction's 17.4% MSE improvement are not reconciled in the provided content.
• Determining when alignment should be a design goal. The 0.33 correlation is described only as a positive trend; a more predictive theory of which datasets benefit would help users decide whether to apply AliO.
• Generalizing the metric. TAM is built on pairwise MAE between overlapping predictions for a single lag parameter; extending it to non-uniform lags or to domains beyond the tested benchmarks is not addressed.

Target Audience

Researchers and practitioners working on long-term time series forecasting who care about model reliability as well as accuracy, especially those deploying models in rolling-forecast settings such as energy, weather, health, and traffic. It is also relevant to anyone evaluating forecasting models who needs a metric that captures consistency between repeated predictions rather than only error against ground truth. Readers should be comfortable with standard LTSF baselines and basic frequency-domain concepts.

Authors’ abstract

Long-term Time Series Forecasting (LTSF) tasks, which leverage the current data sequence as input to predict the future sequence, have become increasingly crucial in real-world applications such as weather forecasting and planning of electricity consumption. However, state-of-the-art LTSF models often fail to achieve prediction output alignment for the same timestamps across lagged input sequences. Instead, these models exhibit low output alignment, resulting in fluctuation in prediction outputs for the same timestamps, undermining the model's reliability. To address this, we propose AliO (Align Outputs), a novel approach designed to improve the output alignment of LTSF models by reducing the discrepancies between prediction outputs for the same timestamps in both the time and frequency domains. To measure output alignment, we introduce a new metric, TAM (Time Alignment Metric), which quantifies the alignment between prediction outputs, whereas existing metrics such as MSE only capture the distance between prediction outputs and ground truths. Experimental results show that AliO effectively improves the output alignment, i.e., up to 58.2% in TAM, while maintaining or enhancing the forecasting performance (up to 27.5%). This improved output alignment increases the reliability of the LTSF models, making them more applicable in real-world scenarios.