We Need a More Robust Classifier: Dual Causal Learning Empowers Domain-Incremental Time Series Classification

AI summary

Overview

Research area: Machine learning for time series classification, specifically domain-incremental continual learning and causal representation learning.

Technical level: Intermediate.

Scope: The paper introduces DualCD, a lightweight dual-causal disentanglement framework that separates class-causal features from spurious features so that time series classifiers stay robust when data arrives as a stream of changing domains.

What This Paper Is About

Time series classifiers are usually trained on a fixed domain, but in real deployments new domains keep arriving over time — new users, new sensors, new subjects — while the label space stays the same. Models trained this way suffer catastrophic forgetting: accuracy on earlier domains collapses as new domains are learned. The authors target this Domain-Incremental Time Series Classification (DI-TSC) setting and argue that the main cause of forgetting is that models latch onto spurious features (intra-class and inter-class confounding) instead of the causal features that genuinely define a class.

Key Contributions

1. Pioneer framing of DI-TSC. The authors state this is the first study to address domain-incremental time series classification, a setting they argue more accurately reflects real-world conditions than task-incremental or class-incremental learning.
2. The DualCD method. A lightweight, model-agnostic framework combining an orthogonal disentanglement module (two negatively correlated soft masks that split temporal representations into causal Z_R and spurious Z_I components) with a dual causal intervention mechanism that constructs variant samples from intra-class and inter-class perturbations.
3. A new benchmark and evaluation metric. The DI-TSC benchmark includes a novel metric, Performance-aware Relative Forgetting (PRF), alongside multiple datasets and comprehensive baselines. PRF adds a performance-dependent modulation factor so that models with very low accuracy cannot appear to "forget less" simply because they had little to forget.
4. Extensive experiments. DualCD is compared against 12 baselines across four datasets and is also plugged into six different time series backbones to demonstrate broad applicability. Source code is available at https://github.com/ZhipengLiu75/DualCD.

Main Findings

• DualCD achieves the best overall performance on all four datasets. Accuracy (ACC) and forgetting scores (RF, PRF) are best for DualCD on HAR, HHAR, ISRUC, and Sleep-EDF, with significance at p < 0.01 based on a t-test over five independent runs.

• Specific headline numbers. DualCD reaches ACC of 0.8565 (HAR), 0.7789 (HHAR), 0.7277 (ISRUC), and 0.7459 (Sleep-EDF); RF of 0.1410, 0.2044, 0.0864, 0.1264; and PRF of 0.0266, 0.0810, 0.0318, 0.0331 respectively.

• Gains over the strongest time series classification baseline (DisMS-TS). Accuracy improvements of 3.94%, 6.72%, 9.00%, and 4.01% across the four datasets; relative forgetting reductions of 10.58%, 20.65%, 12.55%, and 3.52%.

• Gains over the strongest domain-incremental baseline (DualCP). Accuracy improvements of 3.16%, 6.29%, 5.49%, and 3.43%; RF reductions of 9.73%, 18.82%, 9.14%, and 4.24%.

• Low forgetting can be misleading. DLinear achieves a superior RF to all other baselines on Sleep-EDF, but its low accuracy limits practical utility. The authors present PRF as a more reliable selector because it factors in overall performance.

• Every component matters (ablation). Removing the intra-class loss, the inter-class loss, both losses, or replacing disentanglement with two MLPs caused maximum ACC degradations of 9.37%, 8.47%, 14.12%, and 8.37%, and PRF increases of up to 93.75%, 69.13%, 65.43%, and 65.62% respectively.

• Both intervention levels are needed. Extremely large or small values of the trade-off hyperparameter λ (tested over {0.1, 0.3, 0.5, 0.7, 0.9}) do not lead to optimal performance. Best ACC occurs at λ = 0.5 on HAR, HHAR, and ISRUC, and at λ = 0.3 on Sleep-EDF, with a similar pattern for PRF.

• Plug-and-play compatibility. Integrating DualCD into DLinear, PatchTST, iTransformer, xPatch, TimeMixer++, and PatchMLP improved both stability and plasticity on all four datasets. The largest example is DLinear, with up to 66.26% ACC improvement and up to 78.07% PRF reduction; the authors attribute this to DLinear's linear structure being especially prone to superficial features.

• Visualization analysis. The paper compares conventional temporal representations Z from vanilla DLinear against causal representations Z_R from DualCD-enhanced DLinear on the test set of domain 1 from HAR, though the truncated content does not report quantitative details of this visualization.

Methodology in Plain English

The starting point is a standard pipeline: an encoder ψ turns a time series into a temporal representation Z, and a classifier φ maps that representation to a label.

1. Disentangle the representation. A single-layer MLP produces a score vector from Z. Applying a sigmoid to that score gives one mask, and applying a sigmoid to its negation gives a complementary mask. Multiplying Z element-wise by these two negatively correlated masks yields a causal component Z_R and a spurious component Z_I that do not overlap.

2. Intervene on the spurious part (intra-class). For a sample of class i, the authors swap its spurious feature for the spurious feature of another sample of the same class, then add the pieces back together and feed the result to the classifier. The label should not change, because a well-behaved model should ignore spurious features.

3. Intervene with another class's causal part (inter-class). They take the causal feature of a sample from a different class and treat it as the spurious slot, again recombining and predicting the original label. This pushes the model to make causal features unmistakably discriminative.

4. Train with a weighted combination. The objective is λ·L_intra + (1 − λ)·L_inter, both cross-entropy losses on the variant samples, with λ ∈ [0, 1] balancing the two.

5. Continual protocol. For the first domain, parameters are randomly initialized. For each later domain, parameters from the previous domain are loaded and fine-tuned — no stored data from earlier domains (rehearsal-free). Each domain is split 70% / 15% / 15% into train, validation, and test, with the best validation model used to initialize the next domain; all experiments are run five times and averaged.

Why This Matters

Research impact. The paper reframes time series classification as a domain-incremental problem and argues that the classifier, not just the encoder — which prior work tends to emphasize — is where robustness is won or lost. It also contributes a metric (PRF) that corrects a known flaw in Absolute Forgetting and Relative Forgetting, where a weak model can score well simply because it never performed highly on old domains. The released DI-TSC benchmark gives the community a shared evaluation setup.

Real-world applications:

• Human activity recognition from wearables — the running/walking example in the paper shows how step frequency and stride length vary by user group and can be mistaken for class-defining features.
• Sleep staging and EEG monitoring — Sleep-EDF and ISRUC-S3 involve subject-specific signal patterns that must be learned incrementally.
• Medical diagnostics from sensor streams, where new patients or new devices arrive continuously and each represents a distribution shift.
• Web and mobile services that consume streaming sensor data across an evolving user base, which is where the authors situate the problem.

Industry relevance. Because DualCD is rehearsal-free, it avoids storing raw historical data, addressing privacy and storage costs. Because it is model-agnostic and lightweight, it can be added to existing backbones — the compatibility analysis shows improvements across six different architectures without redesigning them.

Future Directions

• Extending beyond classification. The paper is scoped to classification; the same disentanglement-and-intervention idea is untested on forecasting, anomaly detection, and other time series tasks mentioned briefly in the related work.
• Handling the class-incremental case. Labels are fixed across domains in DI-TSC. Whether dual causal intervention still works when new classes appear alongside new domains — where the inter-class sampling step becomes more delicate — remains open.
• Scaling the intervention strategy. The paper shows best results at intermediate λ values, suggesting the two intervention types can conflict. More adaptive schemes than a single global hyperparameter are a natural next question.
• Richer evaluation of causal quality. The paper's visualization comparison between Z_R and Z is qualitative in the provided content; quantitative measures of how disentangled the causal and spurious components actually become would strengthen the causal claims.

Target Audience

This paper is most useful to machine learning researchers working on continual learning, time series analysis, or causal representation learning, and to practitioners building deployed time series systems — wearables, clinical monitoring, industrial sensing — where new data sources or users appear after deployment. Readers with basic familiarity with deep learning classifiers and the forgetting problem will get the most from it; the causal-theory sections assume more background than the method itself requires.

Authors’ abstract

The World Wide Web thrives on intelligent services that rely on accurate time series classification, which has recently witnessed significant progress driven by advances in deep learning. However, existing studies face challenges in domain incremental learning. In this paper, we propose a lightweight and robust dual-causal disentanglement framework (DualCD) to enhance the robustness of models under domain incremental scenarios, which can be seamlessly integrated into time series classification models. Specifically, DualCD first introduces a temporal feature disentanglement module to capture class-causal features and spurious features. The causal features can offer sufficient predictive power to support the classifier in domain incremental learning settings. To accurately capture these causal features, we further design a dual-causal intervention mechanism to eliminate the influence of both intra-class and inter-class confounding features. This mechanism constructs variant samples by combining the current class's causal features with intra-class spurious features and with causal features from other classes. The causal intervention loss encourages the model to accurately predict the labels of these variant samples based solely on the causal features. Extensive experiments on multiple datasets and models demonstrate that DualCD effectively improves performance in domain incremental scenarios. We summarize our rich experiments into a comprehensive benchmark to facilitate research in domain incremental time series classification.