TimeSAE: Causal Sparse Decoding for Faithful Explanations of Black-Box Time Series Models

AI summary

Overview

Research area: Explainable AI (XAI) for time series — specifically post-hoc, model-agnostic explanation of black-box forecasting and classification models, combining sparse autoencoders with causal counterfactual reasoning.

Technical level: Intermediate. The paper assumes familiarity with autoencoders, sparse coding, counterfactual explanations, and basic causal inference (Pearl's hierarchy, interventions), but the core ideas are explained in accessible terms.

Scope in one sentence: TimeSAE learns a sparse, concept-based decomposition of time series inputs whose reconstructions preserve the black-box model's predictions and remain stable under distribution shift, yielding explanations that are causally faithful and compositional.

What This Paper Is About

Most existing methods for explaining black-box time series models identify which time steps or features influenced a prediction, but they break down when the data at explanation time looks different from the data the model was trained on — a common situation in real deployments. The authors build a sparse autoencoder that decomposes a time series into interpretable "concepts," then tests whether intervening on those concepts produces prediction changes that match the model's true causal behavior. The goal is explanations that stay accurate both inside and outside the training distribution, which prior masking-based methods generally fail to achieve.

Key Contributions

1. A concept-based framework (TimeSAE) for black-box time series explanation. Instead of masking raw time steps and features, the method learns an end-to-end overcomplete sparse autoencoder with JumpReLU activations, producing a dictionary of concepts and a decoder that maps concepts back to explanation-embedded instances that stay close to the original data distribution.

2. A theoretical faithfulness guarantee. The authors prove (Theorem 1) that if the autoencoder reconstructs inputs whose model outputs approximate the originals within a bounded error, then the ordering of causal effects across different concept interventions is preserved — a property they call "order-faithfulness." Higher-ranked interventions retain greater causal impact (measured via the Causal Concept Effect, CaCE).

3. Strong empirical results across eight datasets. TimeSAE is evaluated on two synthetic datasets with known ground truth (FreqShapes, SeqComb-UV) and six real-world datasets (ECG, PAM/PAMAP2, ETTh1, ETTh2, and the newly introduced EliteLJ), against eleven baselines including Integrated Gradients, Dynamask, WinIT, TimeX, TimeX++, CounTS, StartGrad, TIMING, and ORTE. It outperforms these baselines on explanation quality and, especially, on out-of-distribution robustness.

4. A new benchmark dataset and open-source library. EliteLJ contains skeleton-based motion capture of long jump athletes with expert annotations of technique phases (run-up, take-off, flight, landing) and qualitative judgments (good vs. bad take-off, correct vs. incorrect landing posture). Code and data are released as TimeSAE-Lib.

Main Findings

• Existing methods degrade sharply under distribution shift. Baselines such as Integrated Gradients and TimeX lose substantial explanation quality when evaluated on shifted data (temporal-resolution shift, cross-domain transfer, cross-dataset transfer), while TimeSAE's degradation from in-distribution to out-of-distribution settings is markedly smaller.

• TimeSAE leads on both explanation accuracy and distributional similarity. Across datasets it achieves the best AUPRC, KDE log-likelihood, and faithfulness score, and the lowest KL divergence and MMD, indicating its explanation-embedded instances stay closer to the original data manifold.

• Representative numbers (ETTh1 in-distribution): TimeSAE reaches AUPRC 0.741 and faithfulness 2.12, versus 0.702/1.71 for TimeX, 0.719/1.78 for TimeX++, and 0.418/1.36 for Integrated Gradients. Under the ETTh1→ETTh2 shift, the gap persists (AUPRC 0.641 and faithfulness 2.09 for TimeSAE, versus 0.611/1.66 for TimeX).

• Cross-domain and cross-population robustness. The advantage holds on Weather→Energy transfer, PAMAP2→OPPORTUNITY human-activity transfer, and the EliteLJ elite-to-intermediate athlete shift, suggesting the learned concepts transfer across related domains.

• Sparser is not always better. Increasing sparsity generally improves interpretability, but excessive sparsity degrades reconstruction fidelity — a trade-off the authors illustrate directly.

• Contrastive supervision is what enforces causal ordering. InfoNCE is used with counterfactuals from the same intervention as positives and different interventions as negatives, aligning the learned representation with the true ranking of causal effects.

• Qualitative comparison against Integrated Gradients. On a 24-hour ETTh1 forecast with a 48-hour history, IG spreads importance broadly across many lagged events, while TimeSAE concentrates on the most temporally relevant regions.

Methodology in Plain English

The method trains a sparse autoencoder on top of a frozen black-box model without accessing its internals. The encoder maps a time series into a sparse set of concept activations; the decoder maps those concepts back into a new time series. Four losses drive training:

• Reconstruction fidelity — the decoded series must resemble the input.
• Sparsity — enforced with JumpReLU activations, where each concept has a learnable threshold; only concepts above their threshold fire, which avoids the "dead concepts" problem seen with ReLU or TopK activations.
• Label fidelity — the black-box model must produce the same prediction for the reconstructed series as for the original.
• Contrastive counterfactual loss — interventions on concepts that should have high causal impact are pushed to produce larger output changes than low-impact interventions.

To handle out-of-distribution inputs, a compositional consistency loss is added: synthetic concept combinations (which may fall outside the training distribution) are decoded and re-encoded, forcing the encoder to remain a reliable inverse of the decoder beyond the training support.

For interpretability at the level of feature groups rather than raw time steps, the decoder is decomposed into a functional ANOVA-style expansion: a bias term, first-order concept contributions, second-order pairwise interactions, and higher-order terms, each gated by a Bernoulli sparsity mask. This lets users see individual concept effects and interactions among them, and generalizes earlier neural ANOVA and GAM decompositions. Finally, learned concepts are aligned with human-understandable labels using Concept Activation Regions with kernel SVR/SVC, which does not assume linear separability — a better fit for time series than linear probes.

Why This Matters

Impact on research. The paper reframes time series explainability as a problem of concept learning plus causal inference, and supplies a formal faithfulness guarantee rather than relying on empirical plausibility alone. It also introduces a benchmark dataset with expert annotations, which is rare in this subfield.

Real-world applications:

• Clinical monitoring: Explaining arrhythmia detection models (e.g., ECG/QRS intervals) where clinicians need to know which signal features drove an alert and whether that reasoning holds for atypical patients.
• Energy grid management: Justifying demand and stability forecasts so operators can act on model outputs even when conditions drift from historical patterns.
• Human activity recognition and wearables: Explaining activity classifiers across different sensor placements, devices, and user populations.
• Sports performance analytics: Using EliteLJ-style pose data to tell athletes which movement phases and postures most influenced an assessed jump quality, with expert labels available to validate the explanation.

Industry relevance. Model-agnostic, post-hoc explanation is attractive to organizations deploying proprietary or third-party time series foundation models (the paper evaluates TimeGPT and Chronos, among others) where internal parameters are inaccessible. Robustness under distribution shift is directly relevant to regulated sectors — finance, healthcare, energy — where explanation quality must persist when the operating environment changes.

Future Directions

• Scaling to larger and longer-horizon foundation models. The paper tests Chronos and TimeGPT but the compositional and causal losses may behave differently with very large pretrained backbones and much longer context windows.
• Extending the causal guarantee beyond ordering. Currently the theory covers preservation of causal-effect ordering under bounded approximation error; absolute-magnitude faithfulness is a stronger and still open target.
• Choosing sparsity and intervention sets automatically. The sparsity-fidelity trade-off is currently empirical, and counterfactuals are obtained without supervision; principled selection procedures would improve usability.
• Human evaluation of concept alignment. Concept labels come from SVR/SVC alignment against a small number of manual definitions; whether human experts find the resulting concepts actionable — particularly on EliteLJ — remains to be studied.

Target Audience

Researchers and practitioners in explainable AI, time series analysis, and causal inference who need to justify predictions from black-box or pretrained sequence models. It is also relevant to domain experts in healthcare, energy, and sports science who consume model explanations, and to engineers building interpretability tooling, since the released TimeSAE-Lib provides a directly usable implementation. Readers without prior exposure to autoencoders or counterfactual reasoning will need to consult the cited background material.

Authors’ abstract

As black box models and pretrained models gain traction in time series applications, understanding and explaining their predictions becomes increasingly vital, especially in high-stakes domains where interpretability and trust are essential. However, most of the existing methods involve only in-distribution explanation, and do not generalize outside the training support, which requires the learning capability of generalization. In this work, we aim to provide a framework to explain black-box models for time series data through the dual lenses of Sparse Autoencoders (SAEs) and causality. We show that many current explanation methods are sensitive to distributional shifts, limiting their effectiveness in real-world scenarios. Building on the concept of Sparse Autoencoder, we introduce TimeSAE, a framework for black-box model explanation. We conduct extensive evaluations of TimeSAE on both synthetic and real-world time series datasets, comparing it to leading baselines. The results, supported by both quantitative metrics and qualitative insights, show that TimeSAE provides more faithful and robust explanations. Our code is available in an easy-to-use library TimeSAE-Lib: https://oublalkhalid.github.io/TimeSAE/