Dynamic Dual-Feedback Conformal Inference for Time Series Forecasting

AI summary

Overview

Research area: Uncertainty quantification for time series forecasting — specifically online conformal inference under non-exchangeability.

Technical level: Advanced (the paper includes regret bounds and coverage theorems, though the core idea is intuitive).

Scope: The paper proposes DDCI, an online conformal method that updates prediction intervals using two opposing feedback signals — one driven by observed miscoverage and one derived from an empirical conformal quantile — to produce narrower and smoother intervals at near-nominal coverage.

What This Paper Is About

Standard conformal prediction assumes exchangeable data, which fails for time series where distributions drift. Existing online fixes (e.g., ACI, OGD, ECI) adjust the interval using only a single binary or error-based signal about whether the last true value was covered — which makes the interval shrink until a miss occurs, then over-expand, producing unstable and unnecessarily wide intervals. DDCI adds a second, counteracting signal taken from the empirical conformal threshold so that updates are damped when the current threshold is already close to a well-calibrated reference, keeping intervals both responsive and stable.

Key Contributions

1. A dual-feedback online conformal method (DDCI): the update rule combines the standard OGD term with an actual feedback term that quantifies the magnitude of the observed error, and an estimated feedback term derived from the empirical threshold of split conformal inference, entering with the opposite sign to form a push-pull control mechanism.
2. An adaptive gating design: the two feedback terms use complementary weights (ρ_t and 1 − ρ_t*), so the estimated feedback dominates when errors are small (damping noise) and vanishes when errors approach the global bound 2B (letting the actual feedback adapt quickly).
3. Theoretical analysis: a bound on the threshold (Proposition 4.3), a dynamic regret bound (Theorem 4.4), and a long-run average miscoverage bound (Theorem 4.6) with an explicit, controllable bias term governed by hyperparameters c and ε.
4. Empirical study: experiments on 3 synthetic and 5 public datasets with 4 forecasting algorithms (Prophet, AR, Theta, Transformer) against ACI, the OGD family, PID control, ECI and COP, plus stability analysis and ablations.

Main Findings

• Narrower intervals at target coverage: Under Prophet on the synthetic datasets, DDCI achieved 89.9% coverage with average width 8.08 ± 0.82 on Synthetic Data 1 (vs. 10.07 ± 1.14 for COP and 11.01 ± 0.50 for OGD); 90.0% with width 13.15 ± 5.35 on Synthetic Data 2; and 89.8% with width 73.33 ± 41.58 on the changepoint setting, where ECI's coverage fell to 81.9% ± 17.84.
• Large width reductions on stock data: On the Google dataset with Prophet, DDCI reported 89.8% coverage, average width 39.20 and Winkler score 52.80, versus 49.72 / 63.75 for COP and 56.07 / 75.46 for ECI. On Amazon with Prophet, DDCI reported width 30.87 and Winkler 41.06; on Microsoft, width 3.26 and Winkler 4.55.
• ACI produces infinite widths: The paper reports that ACI satisfies coverage in theory but yields infinite interval widths on the Google, Amazon and Microsoft datasets across all four forecasting algorithms.
• Better stability: In the MASD stability table under Prophet, full DDCI recorded the lowest values on all five public datasets (GOOGL 6.89, AMZN 5.35, MSFT 0.54, Delhi 0.28, Electricity 0.0603), and achieved the lowest MASD in 19 of 20 dataset–forecaster settings in the appendix results.
• Ablation shows the two feedbacks play distinct roles: Adding the estimated feedback reduced MASD by 2.3% to 33.5% relative to the actual-feedback-only configuration, while changing coverage by at most 0.1 percentage points and mean width by at most 5.5%. Actual feedback alone (ε = 0) was not uniformly smoother than the no-additional-feedback setting (c = 0).
• The quality of the empirical threshold matters: Improvements were less pronounced under the AR model, which the authors attribute to the estimated threshold overestimating local uncertainty when conformal intervals are already conservative.
• Theory shows a controllable bias: The coverage bound carries a non-vanishing O(d) penalty and a bias term (cL_h + εγ)(2B + K₊M_T); the authors argue this is a deliberate bias-variance trade-off, reducible by choosing a small c and small ε, and describe the bound as a worst-case adversarial quantity that is far from tight in practice.
• Why the estimated feedback stabilises: The variance of the empirical quantile is approximated as α(1−α)/(w·[f_t(q_t*)]²), shrinking on the order of O(1/w), whereas the raw score has O(1) variance — so anchoring on q_t* reduces sensitivity to outliers.

Methodology in Plain English

The starting point is standard online conformal inference: maintain a threshold q_t, and after each time step, nudge it up if the true value fell outside the interval and down if it fell inside. That single nudge is coarse — it says nothing about how far outside the value was, and it causes the interval to oscillate.

DDCI adds two shaped correction terms to this nudge. The actual feedback looks at the gap between the current conformal score and the current threshold, scales it by how large that gap is relative to the spread of recent scores, and passes it through a bounded, symmetric squashing function (tanh) so that big misses produce bigger corrections without exploding. The estimated feedback does the same thing but compares the score to a reference threshold q_t* — the (1−α) empirical quantile of the last w scores, computed by ordinary split conformal (a version weighted by 0.99^(t−i) is also tested, following NexCP). Crucially, this second term is subtracted rather than added, and it is weighted by (1 − ρ_t*), so it only matters when the gap to the reference threshold is small; when the gap is large, it fades out and lets the actual feedback take over.

The authors then prove a bound on how far the threshold can drift, a dynamic regret bound against a time-varying optimal threshold sequence, and a long-run miscoverage bound. Empirically, they generate three synthetic series (a random walk with periodic volatility, an exponential growth process, and a changepoint process with breaks at t = 600 and t = 1200) and use five public series (Google, Amazon and Microsoft stock prices, Delhi temperature, and Electricity), running each through Prophet, AR, Theta and Transformer backbones. They target a 10% miscoverage rate, build asymmetric intervals, and evaluate coverage, average width, Winkler score and the mean absolute successive difference (MASD) of interval width.

Why This Matters

Forecast intervals that are valid but enormous are of little practical use, and intervals that are tight but invalid are worse. DDCI offers a way to get closer to the target coverage with materially narrower and smoother intervals, which directly affects downstream decisions that consume those intervals.

Real-world applications:

• Finance: the paper evaluates on Google, Amazon and Microsoft stock price series, where interval width translates into risk-buffer cost.
• Energy: the Electricity dataset targets load forecasting, where over-wide intervals waste reserve capacity and under-wide ones risk outages.
• Meteorology/agriculture: the Delhi temperature dataset represents environmental forecasting with seasonal drift.
• Healthcare and transportation: cited in the introduction as domains where uncertainty quantification is essential for decision-making.

Industry relevance: The method is a drop-in wrapper — it works on top of any black-box forecaster (Prophet, AR, Theta, Transformer in this study), requires no distributional assumptions, and adds only a reference quantile computed from a trailing window. Code is released at https://github.com/sdu1998/DDCI.

Future Directions

• Reduce or eliminate the bias term: The coverage bound carries a non-vanishing O(d) penalty from the extra feedback; tightening this, or making the trade-off adaptive rather than fixed by c and ε, is a natural theoretical next step. The sensitivity analysis of ε is only partially presented in the supplied text.
• Improve the reference threshold q_t:* The paper shows the benefit depends on the quality of q_t* and that gains shrink when the interval is already conservative (e.g., under the AR model) — better or regime-aware estimators of q_t* are an open avenue.
• Learn the hyperparameters: The steepness c, range d, and weight ε are set by hand (ε = 0.2, tanh(x/2) in the main experiments), so online tuning of these values is an obvious extension.
• Extend beyond univariate interval regression: The framework is presented for scalar time series with absolute-error scores; adapting the dual-feedback idea to multivariate forecasting, other score functions, or classification sets is untested here.

Target Audience

Researchers and graduate students working on conformal prediction, uncertainty quantification and online learning, plus practitioners in forecasting-heavy industries (finance, energy, operations) who need calibrated prediction intervals on non-stationary streams and are comfortable implementing a modified online update rule. Readers without a background in conformal inference will find the intuition accessible, but the theory sections require familiarity with regret analysis and quantile loss.

Authors’ abstract

Quantifying uncertainty in time series forecasting is particularly demanding because sequential data exhibit temporal dependence and are prone to distributional changes. Conformal inference has emerged as a powerful uncertainty quantification approach through the construction of prediction sets. Recent advances have introduced online conformal methods that adaptively adjust prediction thresholds through feedback mechanisms. However, the existing feedback mechanism typically relies solely on miscoverage indicators (actual feedback), whether the true label falls within the interval at each time step, while overlooking the empirical prediction threshold (estimated feedback) that is derived from the oracle conformal method. In this paper, we propose Dynamic Dual-feedback Conformal Inference (DDCI), which incorporates a dual-feedback mechanism consisting of actual feedback and estimated feedback. The former drives the primary adjustment of the intervals based on true observations, while the latter dampens excessive expansions or contractions by leveraging empirical thresholds from conformal inference during updates. By balancing these two signals, DDCI achieves more stable and narrower prediction intervals in sequential settings while maintaining near-nominal coverage in practice.