FlowPath: Learning Data-Driven Manifolds with Invertible Flows for Robust Irregularly-sampled Time Series Classification

AI summary

Overview

Research area: Machine learning for irregularly-sampled time series (ISTS) classification, specifically the construction of control paths for Neural Controlled Differential Equations (Neural CDEs) and learnable invertible flows (normalizing flows).

Technical level: Advanced. The paper requires familiarity with Neural ODEs, Neural CDEs, Riemann–Stieltjes integrals, normalizing flows, diffeomorphisms, and generalization bounds via Rademacher complexity.

Scope: This paper proposes FlowPath, a method that replaces fixed interpolation of the Neural CDE control path with a learned, invertible neural flow, and evaluates it on 18 benchmark datasets plus two real-world case studies.

What This Paper Is About

Irregularly-sampled time series are common in practice, but the Neural CDE framework that handles them is highly sensitive to how the control path is built from discrete observations. Existing approaches use fixed interpolants (e.g., linear or cubic splines), which impose geometric assumptions that can misrepresent the underlying data manifold, especially when many observations are missing.

FlowPath's goal is to learn the geometry of the control path itself rather than merely connecting observed points, using an invertible neural flow that enforces information-preserving, well-behaved transformations. The claim is that modeling both the dynamics along the path and the geometry of the path leads to more robust classification.

Key Contributions

1. A learnable control path built from an invertible neural flow. FlowPath defines the control path as Φ(t) = F(t, x; θ_F), where F is a diffeomorphism in its second argument, replacing fixed spline interpolation with a data-adaptive transformation parameterized by a neural network.

2. A framework coupling invertible path construction with Neural CDE dynamics. The hidden state evolves via a Riemann–Stieltjes integral driven by Φ(t), with an equivalent ODE form ż(t) = f(t, z(t); θ_f) Φ̇(t), where numerical solvers treat Φ̇(t) as a known function.

3. Three theoretical properties. Theorem 1 (Preservation of Probability Density) shows the log-density change is driven by the divergence of the controlled dynamics; Theorem 2 (Existence and Uniqueness of Solutions) gives the standard Lipschitz guarantee; Theorem 3 (Generalization Bound) bounds the gap between true and empirical risk by α·(1/√n) + √(ln(1/δ)/2n), with α depending on the loss, L_f, and M_Φ.

4. Extensive empirical validation. Evaluation on 18 UEA & UCR benchmark datasets under regular, 30%, 50%, and 70% missingness, plus the PAMAP2 human activity recognition dataset and the PhysioNet Sepsis medical dataset.

Main Findings

• Highest average accuracy across missingness settings. On 18 benchmark datasets, FlowPath reaches an average accuracy of 0.730 (0.087) with average rank 3.5, compared with the strongest baseline DualDynamics at 0.708 (0.092) and rank 4.8, and Neural CDE at 0.667 (0.075) with rank 7.1. FlowPath is highest in every individual setting: 0.731 (0.083) regular, 0.743 (0.091) at 30% missing, 0.726 (0.084) at 50%, and 0.718 (0.090) at 70%.

• Robustness improves as missingness grows. In the pairwise comparison against DualDynamics, FlowPath records 8 wins / 2 ties / 8 losses in the regular setting but 11/1/6 at 30% missing, 11/2/5 at 50%, and 10/1/7 at 70%, totaling 40 wins / 6 ties / 26 losses across all settings.

• Invertibility matters, not just learnability. The ablation shows Neural CDE at 0.667 (0.075), a non-invertible MLP-based learned path at 0.689 (0.084), and FlowPath at 0.730 (0.087), indicating that structure (invertibility) contributes beyond merely learning the path.

• Learned manifolds align better with ground truth. Qualitative analysis on BasicMotions (four activities, each performed five times, sampled every 0.1 seconds over a 10-second period) shows the non-invertible MLP path overfits sparse points and produces a disordered curve, while FlowPath yields a smoother, more coherent path. 1D and 2D KDE comparisons show FlowPath's distribution aligns more closely with the original than the MLP's.

• Real-world HAR performance. On PAMAP2 (5,333 segments, 8 daily living activities, 17 sensor modalities, three sensors on wrist, chest, and ankle plus a heart rate monitor), with 60% of data points randomly removed, FlowPath with a GRU Flow achieves F1 of 95.6 ± 0.2 at 0% sensor dropout, 88.3 ± 0.4 at 10%, 79.1 ± 1.9 at 20%, 70.0 ± 0.7 at 30%, 59.6 ± 1.0 at 40%, and 54.1 ± 1.0 at 50%. The gains over Neural CDE widen as dropout increases (e.g., 57.9 vs 51.6 at 50%).

• Strong results on the original PAMAP2 sampling. With all sensors under original irregular sampling, FlowPath reports 94.8 ± 0.2 accuracy, 95.8 ± 0.4 precision, 95.5 ± 0.2 recall, and 95.6 ± 0.2 F1, versus Neural CDE at 94.2 ± 0.5, 95.2 ± 0.4, 94.8 ± 0.5, and 95.0 ± 0.4.

• Medical benchmark results. On PhysioNet Sepsis (40,335 patients, 34 temporal variables), AUROC with observation intensity (OI) is 0.919 ± 0.005 (ResNet flow), 0.918 ± 0.005 (GRU), and 0.916 ± 0.006 (Coupling); without OI it is 0.869 ± 0.006, 0.870 ± 0.005, and 0.866 ± 0.002. For comparison, DualDynamics is 0.918 ± 0.003 with OI and 0.873 ± 0.004 without, and Neural GSDE is 0.909 ± 0.001 and 0.884 ± 0.002.

• Moderate computational cost. The performance–computation analysis across flow architectures (ResNet, GRU, Coupling Flow), layer counts n_l, and hidden sizes n_h shows FlowPath variants outperforming Neural CDE at matched parameter counts, with the gap widening under higher missingness.

Methodology in Plain English

The starting point is the observation that a Neural CDE evolves a hidden state in response to a "control path" built from the input time series. Traditionally, that path is drawn by connecting observed points with a fixed rule such as a straight line or a cubic spline. When data are sparse, these fixed rules can draw a curve that does not reflect reality.

FlowPath instead trains a neural network F to draw the curve. The key constraint is that F must be invertible — a diffeomorphism — in its input, a property borrowed from normalizing flows. This means the transformation cannot tear, fold, or collapse information, and every transformation has a well-defined inverse and Jacobian.

Concretely, the control path is Φ(t) = F(t, x; θ_F), where x(0) is the earliest available observation. The hidden state z(t) is then integrated as z(t) = z(0) + ∫ f(τ, z(τ); θ_f) dΦ(τ), with z(0) = h(x(0)) for an embedding layer h. A numerical solver integrates the equivalent ODE form forward in time.

Because invertible flow architectures can approximate any diffeomorphism under suitable conditions, the authors argue that the learned path stays smooth and information-preserving by design, avoiding the instability that unconstrained learned paths can introduce. The training data setup is a dataset of (X_i, y_i) pairs with non-uniform sampling intervals Δt_j where some intervals may be entirely missing.

Why This Matters

Impact on research. The paper reframes a commonly treated hyperparameter — the choice of interpolant in Neural CDEs — as a learnable, structurally constrained component. It connects the ISTS literature with normalizing flows and neural flows, and provides theoretical grounding (density preservation, well-posedness, generalization bound) for that connection. It also offers an explicit distinction between merely learning a path (an MLP) and learning an invertible path, with ablation evidence that the constraint drives much of the gain.

Real-world applications:

• Clinical monitoring and deterioration prediction from sparse electronic health records, as illustrated by the PhysioNet Sepsis evaluation on 40,335 patients with 34 temporal variables.
• Wearable human activity recognition with sensor dropout, as illustrated by PAMAP2 with wrist, chest, ankle, and heart rate sensors across 8 daily living activities.
• Electrocardiogram and electroencephalogram analysis, one of the three dataset categories used in the 18-dataset benchmark.
• Motion and human activity recognition from smartwatch data, as illustrated by the BasicMotions qualitative study.

Industry relevance. Any deployed system ingesting sensor or log data with unreliable sampling — wearables, medical devices, industrial monitoring, remote sensing — faces the same control-path problem. FlowPath's robustness pattern (gains over baselines growing with missingness) is directly relevant to production settings where dropout is common. The paper reports a GitHub repository for code at https://github.com/yongkyung-oh/FlowPath, and the performance–computation analysis suggests the approach is practical at moderate parameter counts.

Future Directions

• Choosing among invertible flow architectures systematically. The paper tests ResNet, GRU, and Coupling Flow configurations and finds FlowPath effective across all three, with the GRU Flow selected for PAMAP2 via hyperparameter tuning. How to select architectures in a principled way rather than by grid search remains open.

• Extending the theory further. The generalization bound depends on α, which in turn depends on the loss, the Lipschitz constant L_f, and the bound M_Φ on the flow derivative. Tightening these dependencies — and relating them to specific flow parameterizations — is a natural next step.

• Broadening beyond classification. The evaluation covers classification accuracy, F1, AUROC, precision, and recall. Whether the learned data-driven manifold benefits regression, forecasting, or generative modeling of irregular series is not reported in this content.

• Scaling and cost characterization. The performance–computation analysis is presented for accuracy versus parameter count under the regular and 50% missing settings. Wall-clock training cost, memory, and behavior on much longer sequences or much higher missingness rates than 70% are not reported.

Target Audience

Researchers and practitioners working on irregularly-sampled time series, Neural ODEs and Neural CDEs, continuous-time deep learning, and medical or sensor-based sequence modeling. It is best suited to readers comfortable with differential equations and normalizing flows; those seeking a plain applied tutorial would need to focus on the experiments section, since the methodology and properties sections are mathematically dense.

Authors’ abstract

Modeling continuous-time dynamics from sparse and irregularly-sampled time series remains a fundamental challenge. Neural controlled differential equations provide a principled framework for such tasks, yet their performance is highly sensitive to the choice of control path constructed from discrete observations. Existing methods commonly employ fixed interpolation schemes, which impose simplistic geometric assumptions that often misrepresent the underlying data manifold, particularly under high missingness. We propose FlowPath, a novel approach that learns the geometry of the control path via an invertible neural flow. Rather than merely connecting observations, FlowPath constructs a continuous and data-adaptive manifold, guided by invertibility constraints that enforce information-preserving and well-behaved transformations. This inductive bias distinguishes FlowPath from prior unconstrained learnable path models. Empirical evaluations on 18 benchmark datasets and a real-world case study demonstrate that FlowPath consistently achieves statistically significant improvements in classification accuracy over baselines using fixed interpolants or non-invertible architectures. These results highlight the importance of modeling not only the dynamics along the path but also the geometry of the path itself, offering a robust and generalizable solution for learning from irregular time series.