Reliable Active Learning from Unreliable Labels via Neural Collapse Geometry

AI summary

Overview

• Research area: Active Learning (AL) for deep learning under unreliable supervision, combining label-efficient data selection with Neural Collapse (NC) geometry.
• Technical level: Advanced — the paper assumes familiarity with active learning acquisition functions, representation learning, and the Neural Collapse literature (simplex equiangular tight frames, terminal phase of training).
• Scope: The paper proposes a single acquisition framework, NCAL-R (also called NCAL), that scores unlabeled candidates using two Neural-Collapse-derived signals, and evaluates it on ImageNet-100, CIFAR100, and CIFAR10 against Random, CoreSet, and CDAL baselines.

What This Paper Is About

Active Learning is supposed to cut annotation cost by picking the most informative unlabeled samples, but its standard heuristics (uncertainty, diversity) can misfire when annotators make mistakes or when the data distribution shifts — they may repeatedly select mislabeled or redundant points. The authors ask whether the geometric regularities that deep networks spontaneously acquire during training (Neural Collapse) can be used as a more reliable selection signal. Their goal is an acquisition rule that improves accuracy with fewer labels, resists synthetic label noise, and generalizes better to out-of-distribution data, without needing auxiliary networks, pseudo-labeling, or task-specific tuning.

Key Contributions

1. Class-Mean Alignment Perturbation (CMAP): A new acquisition score that measures how much adding a candidate sample would perturb the pairwise alignment of class means in feature space. It is derived as the change in Class-Mean Alignment (CMA), the mean pairwise cosine similarity of class means, and simplifies to a dot product, (μ̃̄_t^c − μ̄_t^c)^T (M_t − μ̄_t^c), where M_t is the sum of unit-normalized class means.
2. Feature Fluctuation (FF): A complementary score defined over model checkpoints in the terminal phase of training (T_i to T_f), counting how often the predicted label for a sample flips between consecutive checkpoints. This targets persistent uncertainty even after most features have stabilized.
3. A combined, tuning-free acquisition rule: CMAP and FF are each ranked and z-score standardized, then averaged — Score(x) = (CMAP(x) + FF(x)) / 2 — and the top-k samples are selected, mixing structurally impactful and prediction-unstable points.
4. A broad empirical evaluation: The method is tested on classification, OOD detection, OOD generalization, generalized category discovery, long-tail imbalance, and a transferability ablation against an ActiveOOD method, using a ResNet-18 backbone and 5% acquisition per cycle.

Main Findings

• Higher accuracy at lower label budgets: On ImageNet-100 (reported in Table 1/Table 3 as AUROC scores for Far-OOD detection on OpenImage-O with ImageNet-100 training), NCAL scores 82.49 at the smallest reported budget versus 77.18 (Random), 81.78 (CDAL), and 81.56 (Coreset), and reaches 91.53 at the largest reported NCAL budget, versus Random's 88.67. The 100% data reference AUROC is 93.68.
• OOD detection: The paper reports these Far-OOD AUROC comparisons on OpenImage-O as evidence that Neural Collapse-guided selection improves OOD detection as well as classification; the extracted tables 1 and 3 appear with the same caption and values.
• OOD generalization via linear probe: In Table 4, NCAL reaches an average linear-probe accuracy of 48.98 across eight OOD datasets (ImgNet-R, CIFAR100, Flowers, NINCO, CUB, Aircraft, Pets, STL), compared with 46.95 (Random), 47.21 (CDAL), and 47.00 (Coreset), which the authors describe as roughly a ~2% average improvement. The 100%-data reference average is 50.87.
• Validation accuracy: NCAL achieves 72.11% validation accuracy in the same table, versus 69.51 (Random), 69.09 (CDAL), and 68.65 (Coreset); the 100% reference is 79.16%. NCAL's training accuracy of 95.22 is the lowest among the compared rows.
• Generalized category discovery (label drift): Under the GCD protocol with a 60–40 split, NCAL reaches 35.07 all-class accuracy, 51.95 old-class accuracy, and 23.05 new-class accuracy, with 37.76 validation accuracy. This is +2.1 points on novel classes and +1.6 points on known classes over the best baseline, without supervision on novel classes.
• Better inter-class separation: The density plot of penultimate-layer inter-class distances shows NCAL-R shifted rightward with a mean of 15.944, compared with Random (15.114), Coreset (15.070), and CDAL (15.130).
• Long-tail robustness: On a highly imbalanced ImageNet-100 built with exponential decay of class counts at β = 0.05, producing a pool of 41,454 samples, NCAL-R reaches 45.15% versus 42.30% (Random), 42.06% (Coreset), and 41.94% (CDAL) — an improvement of +3% with only 16k images.
• ActiveOOD transferability ablation: When the OOD filtering component of the recently proposed SISOMe method is removed for a closed-set AL setup, it performs significantly worse than both the standard baselines and NCAL.
• CIFAR100 and CIFAR10: Figure 1 compares test accuracy across label budgets on ImageNet100, CIFAR100, and CIFAR10, with 100% data references of 79.16%, 70.75%, and 90% respectively. Results are averages of 3 independent runs.

Methodology in Plain English

The setup is standard pool-based active learning: a small labeled set and a large unlabeled pool, with a model retrained each cycle and a fixed fraction of the pool queried for labels each time (5% per cycle in most experiments). The novelty is in the scoring function used to decide which points to query.

The first score, CMAP, rests on the observation that in late training, deep network features cluster tightly around per-class means, and those means spread out to be as mutually unaligned as possible. The authors compute the average pairwise cosine similarity between class means, then ask: if this candidate sample were added to the labeled pool with its predicted label, how much would that average change? Because only the mean of the predicted class moves, the change collapses into a simple dot product that can be computed cheaply using class counts, class means, and the sample's penultimate-layer feature. Selecting high-CMAP samples is intended to pull the class means toward less-aligned (more separated) geometry, which the authors connect to a generalization-error bound via weight correlation.

The second score, FF, looks at the model's behavior across the last stretch of training. For each unlabeled sample, it counts how many times the predicted label flips between consecutive checkpoints in the terminal phase. Samples whose predictions keep flip-flopping are treated as genuinely ambiguous and worth labeling.

The two scores are z-score standardized separately and averaged, and the top-k scoring samples form the next batch. Because the method only needs features, class means, class counts, and checkpoint predictions, it introduces no auxiliary network, no pseudo-labeling, and no dataset-specific hyperparameter tuning.

Training protocol details: ImageNet100 starts from a 10% pool (13,000 samples), adds 5% (6,500) per iteration, and stops at 40% pool size; CIFAR100 starts at 10% (5,000 images), adds 5% (2,500) per cycle, and stops at 45%; CIFAR10 starts at 2% (1,000 images) and adds 2% per cycle until 20%. All experiments were run on an A100 GPU with 20 GB memory. Evaluation metrics are all-class accuracy, novel-class accuracy, known-class accuracy, and AUROC for binary OOD detection.

Why This Matters

Impact on research. The paper argues that active learning has been dominated by static heuristics (uncertainty, diversity, representativeness) that ignore the geometric structure emerging during training, and it positions Neural Collapse as an untapped selection signal. It also connects AL to reliability concerns — noisy labels, covariate shift, and label drift — suggesting that acquisition decisions can be made "less brittle" to annotation errors. A second contribution is unification: a single framework addresses OOD generalization and novel-class discovery without OOD filtering, pseudo-labels, or cluster-purity assumptions.

Real-world applications:

• Medical imaging annotation pipelines, where expert labels are scarce, expensive, and occasionally wrong.
• Autonomous driving and robotics datasets, where rare or ambiguous categories and long-tail class imbalance dominate the pool.
• Industrial inspection or content moderation, where the label space keeps evolving and new categories appear after deployment.
• Any deployed model monitoring loop where covariate shift makes previously labeled data less representative.

Industry relevance. The method requires no auxiliary networks, no pseudo-labeling, and no task-specific tuning, and it plugs into any backbone or modality from which feature embeddings can be extracted. Its per-sample cost is described as cheap (class counts, class means, one feature), which matters for practitioners trying to reduce labeling spend on large annotation pipelines. Code is available at https://github.com/Vision-IIITD/NCAL.

Future Directions

• Reducing the NC-regime cost. NCAL-R assumes the model has been trained into Neural Collapse, which can require many epochs depending on dataset and architecture — the authors list this as an efficiency limitation and an open question.
• Scaling to large models. Neural Collapse is not well studied in large-scale models such as LLMs, which are typically trained for only a few epochs; whether NCAL-R's assumptions hold in that regime is explicitly left unexamined and untested.
• Broadening the benchmark suite. The reported experiments cover ImageNet-100, CIFAR100, and CIFAR10 with a ResNet-18 backbone; extending to other architectures and modalities is a natural next step given the method's claimed modality-agnosticism.
• Hardening evaluation of unreliable labels. The paper motivates itself with noisy annotations and distribution shift; the extracted content reports synthetic label noise and long-tail experiments but leaves open how the acquisition rule behaves across noise rates and shifting distributions not covered here.

Target Audience

Researchers and practitioners working on active learning, label-efficient training, noisy-label learning, OOD detection and generalization, and generalized category discovery. It is most useful to readers already comfortable with representation learning and the Neural Collapse literature, and to machine-learning engineers designing real-world annotation pipelines who want a selection criterion that does not require auxiliary models or per-dataset tuning.

Authors’ abstract

Active Learning (AL) promises to reduce annotation cost by prioritizing informative samples, yet its reliability is undermined when labels are noisy or when the data distribution shifts. In practice, annotators make mistakes, rare categories are ambiguous, and conventional AL heuristics (uncertainty, diversity) often amplify such errors by repeatedly selecting mislabeled or redundant samples. We propose Reliable Active Learning via Neural Collapse Geometry (NCAL-R), a framework that leverages the emergent geometric regularities of deep networks to counteract unreliable supervision. Our method introduces two complementary signals: (i) a Class-Mean Alignment Perturbation score, which quantifies how candidate samples structurally stabilize or distort inter-class geometry, and (ii) a Feature Fluctuation score, which captures temporal instability of representations across training checkpoints. By combining these signals, NCAL-R prioritizes samples that both preserve class separation and highlight ambiguous regions, mitigating the effect of noisy or redundant labels. Experiments on ImageNet-100 and CIFAR100 show that NCAL-R consistently outperforms standard AL baselines, achieving higher accuracy with fewer labels, improved robustness under synthetic label noise, and stronger generalization to out-of-distribution data. These results suggest that incorporating geometric reliability criteria into acquisition decisions can make Active Learning less brittle to annotation errors and distribution shifts, a key step toward trustworthy deployment in real-world labeling pipelines. Our code is available at https://github.com/Vision-IIITD/NCAL.