Time Series Forecasting via Direct Per-Step Probability Distribution Modeling

AI summary

Overview

• Research area: Deep learning for multivariate time series forecasting (TSF), specifically probabilistic forecasting and self-supervised multi-branch architectures.
• Technical level: Advanced. The paper assumes familiarity with discrete probability distributions, support sets, quantization error, distribution consistency losses, patching, and channel-independent backbones.
• Scope: One-sentence scope: this paper proposes interPDN (interleaved dual-branch Probability Distribution Network), a lightweight model that emits a discrete per-step probability distribution over a predefined support set and takes its expectation as the forecast, using interleaved dual branches and coarse-scale branches as self-supervised constraints.

What This Paper Is About

Most deep TSF models output a single scalar value per future time step, so they carry no explicit signal about how confident or uncertain that prediction is. Some recent models instead output parameters of an assumed distribution (for example, the mean and variance of a Gaussian), which gives interpretable confidence intervals but forces a prior assumption about the distributional form. This paper recasts the forecasting problem itself: at every step the network directly produces a full discrete probability distribution over a fixed set of candidate values, and the prediction is the expectation of that distribution. The goal is to capture uncertainty without assuming a distribution family, while correcting the quantization problems that come with discretizing a continuous target.

Key Contributions

1. Reframing TSF as direct probability density distribution modeling. Instead of a scalar, the backbone outputs a discrete probability distribution at each time step over a predefined support set, and the final output is the expectation over that distribution. This avoids assuming Gaussian or other parametric forms used by models such as AdaPTS. The idea is credited as inspired by a crowd counting study (Lin et al. 2025) in computer vision.
2. An interleaved dual-branch architecture with self-supervised constraints. Because discretization introduces quantization error and misclassification at bin boundaries (a "boundary effect," Sun et al. 2025), two branches are used with interleaved support sets. Their outputs act as mutual auxiliary self-supervised signals, mitigating anomalies from either branch alone.
3. Bi-scale temporal branches. The interleaved dual-branch design is replicated at a coarse time scale (giving four backbone branches in total, with the same architecture but non-shared parameters), so the model is steered toward long-term trend forecasting rather than overfitting to local detail. Coarse-scale outputs are used only as a self-supervised signal and are not merged into the final prediction.
4. Consistent state-of-the-art results on standardized benchmarks. The authors report that interPDN outperforms all nine baselines on multiple real-world datasets, with the best results on 32 of 45 tasks by MSE and 38 of 45 by MAE.

Main Findings

• First-place counts on the main benchmark: Across 45 forecasting tasks over nine datasets, interPDN ranks first on 32 tasks for MSE and 38 for MAE, i.e. 71.11% of tasks for MSE and 84.44% for MAE. On tasks where it is not first, it still ranks second on eight tasks by MSE and six by MAE.
• Margins over specific baselines: Compared with xPatch, interPDN reports 1.51% lower MAE and 2.44% lower MSE. Compared with iTransformer and PatchTST, reported MSE reductions are 35.15% and 5.31%, and MAE reductions are 20.27% and 7.15%. Against RAFT, AMD and MOMENT, the reported average MAE reduction rates are 13.96%, 4.54% and 15.65% respectively.
• Per-branch-count baseline results: In the same table, AMD holds 4 first-place MSE counts and 2 MAE counts, TimeMixer 6 and 2, PatchTST 3 and 0, MOMENT 1 and 1, DLinear 1 and 1, and xPatch 0 and 1. iTransformer, RAFT and TimesNet hold 0 in both columns as reported.
• Distribution head alone gives limited gains: In the ablation study, replacing a scalar head with a per-step discrete distribution head on a single-branch backbone (SBPDP versus SBSP) improves results on 65% of prediction tasks, but in certain cases degrades predictions.
• Interleaved branches help consistently: The interleaved bi-branch probabilistic model (IBBPDP) outperforms both SBSP and SBPDP across all ablation tasks in both MSE and MAE.
• Coarse-scale constraint alone also helps: Adding a coarse temporal-scale branch as a constraint to a single-branch probabilistic head (BSPDP) yields significant improvements, and the full interPDN (interleaved support sets plus bi-scale scheme) universally outperforms both IBBPDP and BSPDP on all prediction tasks.
• Parameter count alone is not the explanation: A naive four-branch scalar model whose outputs are averaged (4BSP) underperforms even SBSP on 45% of forecasting tasks and significantly underperforms interPDN across all tasks.
• Consistency losses are necessary, not optional: On ETTh1, MSE is highest when the interleaved-branch consistency weights are α = β = 0, and MSE peaks when the cross-scale weight γ = 0. Without these constraints the authors report that the dual branches struggle to achieve complementary prediction and effective self-supervision.
• Experimental stability: The standard deviation of the metrics across all prediction tasks was controlled within 0.003, with each experiment repeated three times. This is the only variance-related figure reported.
• Not reported: The paper does not report training time, inference latency, parameter counts, memory use, or the specific values of the support-set size S, the interval bound B, the downsampling factor k, or the chosen α, β, γ, aside from γ being fixed to 0.1 during the α/β sweep.

Methodology in Plain English

The core swap. Instead of asking the network for one number per future step, the authors ask it for a score for each of many candidate values. Those scores are turned into probabilities with a softmax, and the forecast is the probability-weighted average of the candidates. The candidate values form a fixed "support set" that is the same at every time step.

Choosing the candidates wisely. Uniformly spaced candidates would put many values in the middle of a bin and cause misclassification near edges. Because normalized time series tend to be normal, the authors use the normal cumulative distribution function and its inverse to cut the interval [-B, B] into (S-1) sub-intervals of equal probability, then take the midpoint of each adjacent pair of breakpoints plus the two boundary points ±B as the support set Sp1.

Two interleaved sets. A second support set Sp2 is built from the midpoints between adjacent points of Sp1 plus an additional boundary point. This interleaving means a value sitting near a bin edge for one branch tends to sit near a bin center for the other, reducing boundary error. Two branches with independent parameters and independent output layers each produce their own distribution and expectation.

Combining branches by confidence, not by summation. For each time step the maximum probability across the support set is computed for each branch (e1 and e2). The weight is w = e1 / (e1 + e2), and the fused prediction is the weighted sum w ⊙ Xsp,1 + (1 - w) ⊙ Xsp,2. The authors explicitly contrast this with existing summation-based aggregation.

A lightweight backbone, run four times. Each branch splits the input channel-wise (channel independence), normalizes with RevIN, and separates trend from seasonality using an exponential moving average. The seasonal part goes through patching, a linear layer plus activation, 1D convolution over the patch dimension, a ResNet-style merge, and an MLP decoder. The trend part goes through two linear blocks with average pooling and layer normalization, with no activations. The two streams are concatenated into a 2T-dimensional output per branch. The same backbone is instantiated four times: two fine-scale branches and two coarse-scale branches, all with non-shared parameters.

Coarse-scale supervision. The coarse branches use an output layer projecting to T/k × S rather than T × S (k is a positive integer downsampling factor greater than 1, and T must be divisible by k or padding is applied). The coarse prediction Xsp,c is never added to the final forecast; it only supervises.

The loss stack. The prediction loss L_p follows xPatch: an L1 error averaged over T and C, with an arctangent-based decay function θ(i) that weights errors near the lookback window more heavily than distant steps. On top of that sit two MSE consistency losses — L_f between the fine-scale pair and L_c between the coarse-scale pair — and a cross-scale MSE loss L_t between the coarse prediction and the average-pooled (kernel = k, stride = k) fine prediction. The total is L_p + α·L_f + β·L_c + γ·L_t, with α, β, γ ≥ 0. The authors note that a four-times-larger model on an unchanged small dataset is prone to overfitting and degraded generalization, which is why these self-supervised consistency constraints are needed.

Evaluation setup. Nine datasets: ETTh1, ETTh2, ETTm1, ETTm2, Electricity, Weather, Traffic, Exchange-rate and Illness. Illness uses horizons 24, 36, 48, 60; the other eight use 96, 192, 336, 720. Metrics are MSE and MAE. Nine baselines: xPatch, RAFT, AMD, MOMENT, TimeMixer, iTransformer, TimesNet, PatchTST and DLinear. Code is available at https://github.com/leonardokong486/interPDN.

Why This Matters

Impact on research. The paper offers a third option between scalar regression and parametric distribution heads: model the full discrete density directly, and derive the point forecast by expectation. It also shows that a compact, non-Transformer, non-foundation backbone with a deliberately designed support set and consistency losses can beat Transformer-based and foundation-model baselines on standard benchmarks — a counterweight to the trend toward ever larger pretrained forecasters.

Real-world applications (domains named in the paper, spanning finance, energy, healthcare and transportation):

• Energy and grid operations: Electricity and ETTh1–ETTm2 electricity transformer monitoring data, where a distribution over future load supports capacity and maintenance planning rather than a single guess.
• Finance: the Exchange-rate dataset, where the spread of the predicted distribution is directly relevant to risk-sensitive decisions.
• Meteorology and weather-dependent operations: the Weather dataset.
• Transportation and mobility: the Traffic dataset.
• Healthcare: the Illness dataset, with the short horizons of 24, 36, 48 and 60 that the paper evaluates.

Industry relevance. Because the model outputs a distribution rather than just a scalar, downstream systems gain a confidence signal at no extra inference cost — the max probability per step is already computed for branch fusion. The honest caveat is that the paper reports no latency, memory, or throughput numbers, so deployment cost relative to the nine baselines is not established here.

Future Directions

• Tuning the discretization. The support-set size S, the interval bound B and the downsampling factor k are not reported, and the ablation shows sensitivity to the consistency weights. How sensitive the accuracy–resolution tradeoff is to these choices remains an open question.
• Removing the "four-times-parameters" tax. The design multiplies backbone parameters by four on unchanged dataset sizes and relies on consistency losses to control the resulting overfitting. Whether a shared-parameter or smaller-footprint variant achieves the same benefit is not explored.
• Beyond MSE-based consistency. All three auxiliary losses (L_f, L_c, L_t) use MSE, while the prediction loss L_p uses a weighted L1. Alternatives such as distributional divergences between the two branches' probabilities, rather than their expectations, are left open.
• Extending to foundation-model and zero-shot settings. The authors select MOMENT specifically to test whether a lightweight model can beat a TSF foundation model, but do not test interPDN itself in zero-shot, cross-domain transfer, or long-horizon regimes beyond 720 steps.

Target Audience

Researchers and practitioners working on probabilistic time series forecasting, uncertainty-aware deep learning, and self-supervised multi-branch or multi-scale architectures. It is most useful to readers who already understand support sets, quantization error and consistency losses, and who want an alternative to both scalar regression heads and Gaussian-parameter heads. Readers looking for training-cost, latency or deployment comparisons, or for the exact hyperparameters used, will not find them in this paper.

Authors’ abstract

Deep neural network-based time series prediction models have recently demonstrated superior capabilities in capturing complex temporal dependencies. However, it is challenging for these models to account for uncertainty associated with their predictions, because they directly output scalar values at each time step. To address such a challenge, we propose a novel model named interleaved dual-branch Probability Distribution Network (interPDN), which directly constructs discrete probability distributions per step instead of a scalar. The regression output at each time step is derived by computing the expectation of the predictive distribution on a predefined support set. To mitigate prediction anomalies, a dual-branch architecture is introduced with interleaved support sets, augmented by coarse temporal-scale branches for long-term trend forecasting. Outputs from another branch are treated as auxiliary signals to impose self-supervised consistency constraints on the current branch's prediction. Extensive experiments on multiple real-world datasets demonstrate the superior performance of interPDN.