DeMa: Dual-Path Delay-Aware Mamba for Efficient Multivariate Time Series Analysis

AI summary

Overview

Research area: Efficient deep learning architectures for multivariate time series (MTS) analysis, specifically Mamba/state-space-model backbones as linear-time alternatives to Transformers.

Technical level: Advanced. The paper assumes familiarity with Transformer attention, state space models (SSMs), Mamba and Mamba-2, Structured State Space Duality (SSD), linear attention, and frequency-domain (FFT) decomposition.

Scope: The paper proposes DeMa, a dual-path, delay-aware Mamba backbone that separates intra-series temporal modeling from delay-aware cross-variate interaction modeling to achieve linear-time complexity across five multivariate time series tasks.

What This Paper Is About

Transformers dominate multivariate time series modeling because attention captures pairwise dependencies, but their cost grows quadratically with sequence length, which limits long-horizon and large-scale deployment. Mamba offers linear-time sequence modeling, yet the authors argue that applying vanilla Mamba directly to MTS is suboptimal for three reasons: it lacks explicit cross-variate modeling, it struggles to disentangle intra-series temporal dynamics from inter-series interactions, and it does not explicitly model lagged (delay) effects between variables. DeMa's goal is to keep Mamba's linear-complexity advantage while explicitly addressing all three limitations.

Key Contributions

1. A dual-path Mamba backbone (DeMa). The model jointly captures intra-series temporal dynamics through a temporal path and delay-aware cross-variate dependencies through a variate path, targeting a favorable accuracy-efficiency trade-off for long-term and large-scale MTS modeling.

2. Mamba-SSD for intra-series temporal dependencies. A module that serializes each variate along the temporal axis into a 1D sequence, accommodates variable-length inputs, and enables series-independent parallel computation via block-based matrix multiplication (Structured State Space Duality).

3. Mamba-DALA for cross-variate dependencies. A module integrating Delay-Aware Linear Attention (DALA) that models cross-variate interactions using both a global correlation delay and a token-level relative delay, supporting fine-grained, delay-sensitive dependency learning.

4. Adaptive Fourier Filter (AFF) decomposition with theoretical justification. The input is split via FFT into a Cross-Time Component (globally dominant frequencies) and a Cross-Variate Component (residual frequencies). Theorem 1 proves that two non-orthogonal components cannot be represented on disjoint subsets of an orthogonal basis, motivating explicit spectral decomposition before representation learning.

5. Extensive evaluation across five tasks. Long- and short-term forecasting, data imputation, anomaly detection, and series classification.

Main Findings

• Claimed state-of-the-art results across five tasks: The authors report that DeMa achieves state-of-the-art performance on long-term forecasting, short-term forecasting, data imputation, anomaly detection, and series classification. Specific numerical results, dataset names, and baseline lists are not included in the provided paper content.

• Computational efficiency gains are claimed but not quantified here: The abstract and contributions state that DeMa "significantly reduc[es] training time and GPU memory usage." No specific timing or memory figures appear in the provided content.

• Complexity ordering: Figure 1(b) compares complexity across architectures. Transformer: O(T²D); PatchTST and Timer: O(NL²D); Moirai: O(N²L²D); iTransformer: O(N²D); DeMa: O(2NLD²). The paper notes that in typical long-horizon settings T > L ≫ N, D.

• Theoretical result on decomposition (Theorem 1): If the Cross-Time component and the Cross-Variate component are non-orthogonal (⟨X̃, X̂⟩ ≠ 0), then their supports on any orthogonal basis must overlap. The authors argue that even though HiPPO-based SSMs tend to learn approximately orthogonal transformations, a single Mamba applied to raw MTS can still entangle these factors — motivating AFF.

• Design rationale for Mamba-2/SSD: Mamba-2 reformulates SSMs as semi-separable matrices, enabling block-decomposed matrix multiplication. The paper states Mamba-2 achieves a 2–8× faster training process than Mamba-1's parallel associative scan while remaining competitive with Transformers.

• Lag effects are central to the argument: The paper illustrates this with traffic forecasting — an accident in one region may not immediately affect adjacent areas, but its impact often propagates through the network over several minutes. Recurrent hidden-state updates in Mamba depend primarily on the immediately preceding timestep, which the authors argue limits explicit lag modeling.

Methodology in Plain English

The approach is challenge-driven, with one design decision responding to each of the three stated limitations.

First, the raw multivariate series is passed through an Adaptive Fourier Filter. The series is transformed with the Fast Fourier Transform along the time axis for each variable, frequency indices (0 to ⌊T/2⌋) are ranked by averaged amplitude, and the top θ fraction is separated from the remainder. The top-frequency part becomes the Cross-Time Component (broadly shared, long-term structure), and the residual becomes the Cross-Variate Component (window-specific, higher-frequency variation). This pre-splits the data so the two kinds of structure are not modeled by the same path.

Next, both components are tokenized into patch tokens. Each variable is normalized with reversible instance normalization (RevIN), split into L contiguous patches of length P, and each patch is mapped to a D-dimensional embedding by a lightweight 1-D convolutional encoder. The two components are then arranged with two different scan orders: the Cross-Time component keeps temporal order and stacks variables in parallel (N × L × D), while the Cross-Variate component groups tokens by patch position and scans across variables (L × N × D).

These scanned sequences feed stacked DuoMNet blocks, each containing two parallel paths, and each path is coupled with a scan operator and a lightweight Mamba-based module. The temporal path runs Mamba-SSD, which uses two-branch gating and local mixing (a 1-D convolution along the token axis), generates token-dependent SSM parameters (Δ, B, C) with Δ passed through softplus, discretizes a negative-diagonal transition matrix, and performs the SSD operator block-diagonally across variables. The hidden state is zeroed at the start of each series so no cross-series information flows through this path.

In parallel, the variate path runs Mamba-DALA, which combines delay-aware linear attention with the Mamba formulation to capture cross-variate dependencies using both a global correlation delay and a token-level relative delay.

Finally, the two path outputs are combined through a weighted fusion layer and projected to task-specific outputs (forecasting, imputation, classification, anomaly detection) with lightweight heads.

Why This Matters

Impact on research. The paper adds to the line of work replacing quadratic attention with linear-time state-space backbones for time series. Its distinctive argument is that linear efficiency alone is insufficient — a Mamba backbone also needs an explicit mechanism for cross-variate interaction and lagged effects. It also contributes a negative theoretical result (Theorem 1) about when orthogonal-basis separation of entangled components cannot succeed, which constrains design choices for decomposition-based models.

Real-world applications named in the paper:

• Traffic flow forecasting for transportation scheduling.
• Anomaly detection for industrial maintenance.
• Missing data imputation for web stream processing.
• Trajectory classification for health monitoring.

Industry relevance. The stated motivation is scalability and practical deployment for long-term, large-scale MTS modeling, where quadratic computational cost and memory overhead are the bottleneck. The paper positions DeMa as offering reduced training time and GPU memory usage relative to attention-based alternatives, which matters for sensor networks that record measurements continuously at scale. Specific cost figures are not reported in the provided content.

Future Directions

• Quantifying the efficiency claim. The paper reports efficiency gains qualitatively in the abstract and contributions but the provided content gives no measured runtimes, memory figures, or hardware details. Independent replication would be needed to establish the magnitude at different N, L, and T.

• Sensitivity of the AFF threshold θ. The Adaptive Fourier Filter uses a top-θ frequency fraction, but the provided content does not report how θ is selected or how sensitive results are to it, nor whether θ is learned or fixed.

• Extending the delay model. DALA captures a global correlation delay and a token-level relative delay. Whether richer or non-stationary delay structures (for example, delays that change regime over time) can be modeled within the same linear-time framework is an open question.

• Scope beyond the five evaluated tasks. The framework is described as task-agnostic with lightweight task heads. Whether the shared representation transfers to tasks outside forecasting, imputation, anomaly detection, and classification is not addressed in the provided content.

Target Audience

Researchers and practitioners working on time series foundation models and efficient sequence architectures — particularly those already familiar with Transformers, Mamba/Mamba-2, and state space models. It is also relevant to engineers deploying forecasting, imputation, anomaly detection, or classification systems on large sensor networks where memory and compute budgets are constrained. Readers without a background in SSMs or linear attention will find the preliminaries section necessary before the method is accessible.

Authors’ abstract

Accurate and efficient multivariate time series (MTS) analysis is increasingly critical for a wide range of intelligent applications. Within this realm, Transformers have emerged as the predominant architecture due to their strong ability to capture pairwise dependencies. However, Transformer-based models suffer from quadratic computational complexity and high memory overhead, limiting their scalability and practical deployment in long-term and large-scale MTS modeling. Recently, Mamba has emerged as a promising linear-time alternative with high expressiveness. Nevertheless, directly applying vanilla Mamba to MTS remains suboptimal due to three key limitations: (i) the lack of explicit cross-variate modeling, (ii) difficulty in disentangling the entangled intra-series temporal dynamics and inter-series interactions, and (iii) insufficient modeling of latent time-lag interaction effects. These issues constrain its effectiveness across diverse MTS tasks. To address these challenges, we propose DeMa, a dual-path delay-aware Mamba backbone. DeMa preserves Mamba's linear-complexity advantage while substantially improving its suitability for MTS settings. Specifically, DeMa introduces three key innovations: (i) it decomposes the MTS into intra-series temporal dynamics and inter-series interactions; (ii) it develops a temporal path with a Mamba-SSD module to capture long-range dynamics within each individual series, enabling series-independent, parallel computation; and (iii) it designs a variate path with a Mamba-DALA module that integrates delay-aware linear attention to model cross-variate dependencies. Extensive experiments on five representative tasks, long- and short-term forecasting, data imputation, anomaly detection, and series classification, demonstrate that DeMa achieves state-of-the-art performance while delivering remarkable computational efficiency.