Resilient Class-Incremental Learning: on the Interplay of Drifting, Unlabelled and Imbalanced Data Streams

AI summary

Overview

• Research area: Online/streaming machine learning, specifically class-incremental learning (CIL) in non-stationary environments, combined with unsupervised learning, novel-class discovery, and class-imbalanced multi-class classification.
• Technical level: Advanced. The paper is built on formal definitions of concept drift and class imbalance, autoencoder/MLP architectures, reconstruction-loss thresholds, Mahalanobis-distance-based correction, and SMOTE oversampling.
• Scope (one sentence): The paper proposes SCIL, a unified unsupervised streaming framework that simultaneously handles concept drift, class imbalance, label unavailability, and the emergence of new classes, and evaluates it on synthetic and real-world data streams.

What This Paper Is About

Real-world data streams from domains such as critical infrastructure, industrial systems, healthcare, and IoT networks are rarely stationary, rarely labelled, and rarely balanced: anomaly classes are far rarer than normal operation. These problems do not occur in isolation — drift blurs class boundaries, class imbalance biases decisions, missing labels prevent correction, and unseen classes appear without warning. Most existing methods tackle these challenges separately, and the authors note that the latest CIL survey identifies imbalance, weak supervision, and concept drift as an underexplored combination; SCIL is designed to address all of them at once under an unsupervised, one-by-one online learning setting.

Key Contributions

1. SCIL framework. A resilient architecture that combines an autoencoder (AE) with a multi-layer perceptron (MLP) for multi-class classification and new-class detection. The AE's latent representation feeds the MLP, and the whole model is trained under a single dual-loss objective that mixes reconstruction loss and classification loss via a weighting term α ∈ (0, 1).

2. Pseudo-label oversampling with reliability-aware correction. An online oversampling strategy that uses previous model predictions as pseudo-labels for updates, combined with a correction mechanism that uses class density, class scale, and distance (via geometric median and Mahalanobis distance) to purify minority-class queues before resampling.

3. Dynamic replay-based memory. A queue-based storage mechanism that adapts to non-stationary environments, supports training a new model when a novel class is confirmed, and mitigates catastrophic forgetting by revisiting recent examples from all classes.

4. Extensive evaluation and public release. Ablation studies and comparisons against state-of-the-art methods on a mix of real-world and synthetic datasets, plus a computational analysis of memory and time complexity. Datasets and code are released at https://github.com/Jin000001/SCIL.

Main Findings

• The four challenges are entangled, not independent. The paper argues that concept drift blurs class boundaries, imbalance biases decisions toward outdated boundaries, label scarcity blocks error correction, and weak new-class detection produces overconfident mispredictions of novel events — so handling them separately is insufficient.

• A concrete research gap is identified. Most drift-handling methods are supervised; most multi-class methods addressing both drift and imbalance are limited to binary classification; among the reviewed methods only LC-INC explicitly considers class imbalance, and it does not handle drift in an unsupervised manner. Methods such as CPOCEDS, MINAS, SNDProb, and an ensemble approach handle drift without addressing imbalance.

• New-class detection is driven by reconstruction error. An instance is flagged as novel when its reconstruction loss exceeds the threshold θᵢ of the predicted class i. Thresholds are set as the maximum reconstruction loss observed in the corresponding class queue (size m for the majority class, size l for minority classes).

• Correction is applied selectively. The correction mechanism runs only when a new model is created, because frequent correction risks filtering out drifted instances and erasing historical information.

• Memory is bounded. SCIL uses n + 2 queues (n minority classes, one majority class, one novel class) with space complexity O(m + (n + 1) × l), where m and l are fixed and l is small (the paper gives an example of l = 10). This satisfies the fixed-memory property desired for learning in non-stationary environments; memory grows by O(l) only when a new class is confirmed.

• Training triggers are bounded and event-driven. Incremental training runs at intervals t_train (example given: every 2000 steps) with a small number of epochs per session (example given: 10). Training time complexity is O((n + 1) × m), dominated by the queues; rare classes that appear infrequently do not significantly raise training cost, and rarely appearing classes can be forgotten after long inactivity.

• Evaluation spans nine datasets with deliberately varied properties. Synthetic: Sea (2 features, 15000 data points, 2 initial classes, 4 total, 1.3% imbalance rate), Vib (10 features, 15000, 2 initial, 4 total, 2.0%), Blob (3 features, 15000, 2 initial, 7 total, 2.0%). Real-world: WDN (1 feature, 8640 points, 1 initial class, 3 total, 1.2%), MNIST (784 features, 10000, 2 initial, 10 total, 4.5%), KDD99 (116 features, 15000, 2 initial, 4 total, 2.4%), Forest (52 features, 15000, 2 initial, 4 total, 2.4%), Sensorless (48 features, 5000, 2 initial, 3 total, 4.0%), and Shuttle (9 features, 55827, 2 initial, 4 total, 21.9%).

• All three drift forms are represented. Abrupt drift (Blob at time 7000, WDN at 4000, MNIST at 5000, Forest at 10000), recurrent and incremental drift (Sea at 5000 and 10000; Vib at 5000–5050 and 10000–10050; KDD99 at 5000 and 10000), gradual drift (Sensorless at 2500–2600), and unknown drift (Shuttle, listed as unknown in all columns).

• The WDN dataset is a concrete anomaly-detection scenario. It uses chlorine sensor data from a single node, six months of 30-minute samples (8640 points) simulated with WNTR on the Hanoi water distribution network with detection at node 9, where contamination events at different nodes form minority classes and concept drift comes from a sensor offset randomly scaled 1.3–1.5×.

• Quantitative comparison results are not available in the supplied content. The paper claims "significant performance gains in class-imbalanced incremental new-class scenarios under nonstationary environments," but the results section (Sec. 6) and the remainder of the methods table (which begins with Baseline, iForest+MULTI, LOF+MULTI, MINAS, CPOCEDS, SNDProb, KNNENS, OCGCD, UDOR) are truncated, so no accuracy, F1, or other metric values can be reported here.

Methodology in Plain English

The setting is one-by-one online learning: at each time step a single unlabelled input arrives and the model must output a predicted label.

Two-part model. An autoencoder compresses the input into a lower-dimensional latent embedding and reconstructs it; the embedding is fed to a multi-layer perceptron that performs multi-class classification. The whole thing is trained with a combined loss that weights reconstruction against classification, so the representation stays informative enough to react to unseen inputs while the classifier stays discriminative for known classes.

Offline warm-up. The model is pretrained on labelled initial data: a majority class and a small number of minority class examples (the paper gives 10 per class as an example). Novelty thresholds are computed per class as the maximum reconstruction loss seen in that class's queue.

Streaming loop. Each new instance is classified. If its reconstruction loss exceeds the threshold of the predicted class, it is treated as a new class and placed in a separate novel-class queue; otherwise it is appended to the queue of the predicted class. Class queues are continuously updated with the most recent instances, and the majority queue is larger than the minority queues because of imbalance.

Pseudo-labels. Since labels are unavailable online, the previous model's predictions serve as surrogate labels for updates, with a correction mechanism intended to limit error propagation.

Correction before resampling. When a new model is about to be created, each minority queue is purified: the method estimates local density (inverse median k-nearest-neighbour distance), class scale (mean distance to the geometric median), combines them into a relative density compensation term, and derives a retention ratio by convex mixing of normalized density and normalized class size between keep_min and keep_max. The most reliable core subset — those closest to the robust centre under Mahalanobis distance — is retained, and only that subset is oversampled.

Oversampling. SMOTE generates synthetic minority samples by interpolating between a minority instance and one of its K nearest neighbours in the queue.

Triggering retraining. Two events cause training: (a) the novel-class queue reaching capacity l, at which point the class count increments, the novel queue becomes a new class queue, correction is applied, and a new model is trained on SMOTE-resampled queues; and (b) the passage of t_train steps since the last training, which triggers incremental training and threshold updates to counteract drift.

Why This Matters

Impact on research. The paper targets a combination that the CIL literature has explicitly flagged as underexplored — imbalance, weak supervision, and concept drift together — in the multi-class setting rather than the binary anomaly-detection setting where most prior drift-and-imbalance work lives. It also supplies a fixed-memory design and a released codebase, both of which make the problem easier to benchmark reproducibly.

Real-world applications mentioned by the paper:

• Water distribution systems. Detecting contamination events (as in the WDN dataset) where normal operation dominates and contamination at different nodes forms distinct minority classes subject to sensor drift.
• Cybersecurity and network intrusion detection. Distinguishing rare attack types (e.g., smurf, neptune, satan classes in KDD99) from normal traffic as attack patterns and feature distributions shift over time.
• Industrial inspection and manufacturing. Fault diagnosis from equipment vibration data (Vib) and electric drive signals (Sensorless), where defective conditions are rare, sensor behaviour drifts, and new defect types emerge.
• Predictive maintenance and safety-critical monitoring. Catching emerging faults that would otherwise go undetected, in domains where continuous adaptation risks catastrophically forgetting previously learned fault signatures.

Industry relevance. The demands SCIL is designed around — bounded memory, no labels during operation, resilience to drift, and detection of previously unseen failure modes — match the operating constraints of deployed monitoring systems in critical infrastructure and industrial automation, where annotation is impractical and the cost of a missed anomaly is high.

Future Directions

• Reconciling the correction mechanism with drift. The paper deliberately applies correction only at new-model creation because frequent correction risks filtering out drifted instances. Deciding when correction helps versus harms under drift remains an open design question the paper raises.

• Closing the supervision gap in CIL. The authors note that most current CIL methods are fully supervised and trained offline, while most drift methods are supervised; extending unsupervised, stream-native methods beyond the settings studied here is an explicit open direction.

• Scaling to large numbers of incremental classes. Training time complexity is O((n + 1) × m), growing with the number of minority classes and driven by queue size m; how the approach behaves with many emerging classes or very high-dimensional inputs is not resolved in the available content.

• Handling unknown drift. The Shuttle dataset is listed with unknown drift timing and nature, which tests robustness but also highlights that knowing when and how drift occurs is not always available — an open problem for active-style adaptation.

Target Audience

Researchers and practitioners working on streaming and online learning, class-incremental learning, continual learning without forgetting, anomaly and novelty detection, and imbalanced multi-class classification. It is also relevant to engineers deploying monitoring or fault-diagnosis systems in water networks, cybersecurity, industrial manufacturing, and other safety-critical infrastructure, and to readers who want a single reference that frames concept drift, label scarcity, class imbalance, and new-class emergence as one coupled problem.

Authors’ abstract

In today's connected world, the generation of massive streaming data across diverse domains has become commonplace. In the presence of concept drift, class imbalance, label scarcity, and new class emergence, they jointly degrade representation stability, bias learning toward outdated distributions, and reduce the resilience and reliability of detection in dynamic environments. This paper proposes SCIL (Streaming Class-Incremental Learning) to address these challenges. The SCIL framework integrates an autoencoder (AE) with a multi-layer perceptron for multi-class prediction, uses a dual-loss strategy (classification and reconstruction) for prediction and new class detection, employs corrected pseudo-labels for online training, manages classes with queues, and applies oversampling to handle imbalance. The rationale behind the method's structure is elucidated through ablation studies and a comprehensive experimental evaluation is performed using both real-world and synthetic datasets that feature class imbalance, incremental classes, and concept drifts. Our results demonstrate that SCIL outperforms strong baselines and state-of-the-art methods. Based on our commitment to Open Science, we make our code and datasets available to the community.