Sampling Control for Imbalanced Calibration in Semi-Supervised Learning

AI summary

Overview

Research area: Semi-supervised learning (SSL) with class imbalance — specifically class-imbalanced semi-supervised learning (CISSL), where the class distribution of unlabeled data may differ from, or be unknown relative to, the labeled data.

Technical level: Intermediate. The paper is readable with a background in deep learning and classification losses, though it includes a theoretical derivation (Theorem 0.1) and relies on concepts such as the expansion–separation assumption, logit adjustment, and balanced softmax.

Scope: One sentence — the paper proposes SC-SSL, a framework that reduces class imbalance in semi-supervised learning through decoupled sampling control at training time (an expansive classifier) and a bias-vector correction at inference time, and evaluates it on CIFAR10-LT, CIFAR100-LT, STL10-LT, and ImageNet-127.

What This Paper Is About

Existing class-imbalanced semi-supervised learning methods usually correct model predictions by adjusting logits using an estimated class distribution of the unlabeled data. The authors argue this treats imbalance too coarsely, because it blends genuine data imbalance with bias that comes from classes simply being harder to learn, leaving feature-level bias in place.

The goal of SC-SSL is to attack both levels of bias: the bias inside the learned features, and the bias in the final classifier logits. The authors do this by controlling which pseudo-labels get sampled during training, and then by subtracting an "optimization bias vector" from the logits at inference.

Key Contributions

1. A unified framework (SC-SSL) that mitigates both feature-level bias and logits-level bias in imbalanced settings through decoupled sampling control, using an additional expansive classifier guided by the expansion–separation assumption.
2. A theoretical analysis of pseudo-label sampling under class imbalance, identifying the key control factors (γ_u, Δp, and ρ) and using them to dynamically adjust sampling probabilities during training.
3. An inference-time calibration mechanism that treats the linear classifier's bias term as an "optimization bias vector" and subtracts it from the final logits, and that also reuses this bias estimate as a prior to approximate the unlabeled class distribution before training.
4. State-of-the-art results reported on CIFAR10-LT, CIFAR100-LT, STL10-LT, and ImageNet-127 across multiple unlabeled-data distributions, with code released at https://github.com/Sheldon04/SC-SSL.

Main Findings

• Feature attention improves for non-head classes: A Grad-CAM visualization (Figure 1) comparing uniform sampling against SC-SSL shows that attention regions for a non-head class are "significantly improved, capturing more relevant features" under SC-SSL.

• The bias term isolates optimization imbalance: The authors observe that under random sampling, head classes tend to have higher bias values (reflecting data imbalance); under the expansive classifier's sampling, tail classes show higher bias values (an overcorrection); while the output classifier, trained with balanced sampling for inference, has bias terms that exclude data imbalance and thus isolate optimization bias (Figure 3, illustrated on the Consist setting).

• Training-time sampling can be steered by three factors: Pseudo-label sampling probability is stated to be primarily influenced by γ (the class prior); the logit adjustment amount Δp can mitigate or even reverse the effect of data imbalance; and the confidence threshold ρ also affects sampling, with the caution that lower sampling probabilities can lead to confirmation bias as β increases.

• Gains on CIFAR10-LT and CIFAR100-LT (Table 1): With WRN-28-2 trained from scratch, SC-SSL reaches 86.53 (±0.16), 83.89 (±0.35), 89.97 (±0.20), 86.02 (±0.09), 60.65 (±0.14), 57.88 (±0.26), 62.99 (±0.42), and 60.27 (±0.14) across the eight reported configurations, exceeding the strongest listed prior baseline ACR (for example 86.53 vs. 84.10, and 89.97 vs. 89.46).

• Results on uniform, Gaussian, and unknown distributions (Table 2): SC-SSL reports 93.79 (±0.22), 86.45 (±0.02), 93.33 (±0.19), 83.11 (±0.17), 90.84 (±0.29), 91.25 (±0.34), 79.26 (±0.31), and 77.11 (±0.35) on CIFAR10-LT and STL10-LT. On the first of these columns CPE reports 93.81 (±0.14), marginally above SC-SSL's 93.79 (±0.22); SC-SSL is highest on the remaining columns shown.

• ImageNet-127 (Table 3): SC-SSL achieves 62.3 at 32×32 resolution and 69.4 at 64×64, against FixMatch 35.7 / 44.0, CDMAD 55.6 / 61.7, ACR 57.2 / 63.6, CPE 57.8 / 64.1, and SimPro 59.4 / 67.2.

• Head / non-head split is sufficient: Comparing 2, 3, and 4 class intervals (Table 4), results are stable — 83.89 and 86.02 for 2 partitions, 83.54 and 85.98 for 3, and 83.50 and 86.15 for 4 — supporting the coarse two-group split.

• Chaotic distributions (Table 5, N:M = 500:4000): On four chaotic distributions, SC-SSL reports 84.2, 83.1, 85.1, and 85.2, compared with ACR (82.9, 82.7, 83.4, 83.1) and CPE (82.0, 81.4, 82.7, 81.9).

• All three factors matter (Table 6, N:M = 1500:3000): With initialized thresholds the method reaches 86.53 on Consist and 88.54 on Inverse (c-init (2,4), c = 4), while fixed-threshold variants fall to 84.70 / 86.90 (fix-max) and 79.04 / 86.05 (fix-min).

Methodology in Plain English

The method starts by sorting classes by labeled sample count and splitting them into "head" classes (the first half) and "non-head" classes (the latter half). Labeled and unlabeled data live in the same label space, but the unlabeled distribution is treated as unknown.

Three classifiers share one backbone network: the base classifier from the underlying SSL algorithm (FixMatch in the experiments, run on the USB codebase), an output classifier used for the final predictions, and a new expansive classifier. The expansive classifier exists because, as the authors explain, the balanced output classifier cannot have its logit adjustment changed without harming accuracy, and the original classifier can only raise non-head sampling by lowering its confidence threshold, which contradicts the separation assumption under strong augmentation.

The authors analyze pseudo-label sampling through a simplified binary Gaussian problem where the positive class has probability γ ∈ (1/2, 1). They derive an explicit expression (Theorem 0.1) for the probability that a pseudo-label is +1, −1, or masked, in terms of γ, the threshold ρ, the inverse temperature β, the logit adjustment Δp, and the class-conditional standard deviations. From this they conclude that sampling should be controlled by γ, Δp, and ρ, and they connect ρ's initialization to the expansion factor c from the (a, c)-expansion assumption, using separate initialization rules for the balanced classifier and the expansive classifier over non-head classes.

At inference, the authors note that the output classifier is trained with balanced sampling, so its learned bias term reflects only optimization-induced imbalance. They subtract this bias vector from the logits, which is equivalent to dropping the bias term entirely in their formulation. The same bias prior is used before training as a rough estimate of the unlabeled class distribution: after several epochs of estimation training, per-class counts N^e are compared against O predefined anchor distributions rescaled to match, and the closest one by KL divergence determines the initial sampling settings (including c).

Experimental setup details: batch sizes are 64 labeled and 128 unlabeled; CIFAR10-LT, CIFAR100-LT, and STL10-LT images are resized to 32×32 with a WRN-28-2 backbone and no pre-training; ImageNet-127 images are downsampled to 64×64 and 32×32 with a ResNet-50 backbone. τ_b and τ_e are 2 and 4; c is set to 4, 5, 6, 4, and 6 for the consist, uniform, inverse, Gaussian, and inverse-Gaussian anchors respectively; ρ_max is 0.95, with α = 0.005 and ν = 1.0. The unlabeled loss ratio for the balanced and expansive classifiers is 2, and training uses SGD with a fixed learning rate of 0.03, momentum 0.9, and weight decay 0.0005.

Why This Matters

Impact on research: The paper reframes imbalance correction as two separable problems — feature-level bias, addressed through controlled pseudo-label sampling, and optimization-level bias, addressed through the classifier bias term. It also argues against the conservative practice of discarding noisy tail-class pseudo-labels, invoking the expansion assumption and the pseudo-label denoising bound (2c/(c−3))·μ as justification that noisy pseudo-labels still help when the model is robust to augmentation.

Real-world applications (the paper motivates these settings but evaluates only on image benchmarks):

• Continuously collected data streams where the unlabeled class mix drifts away from the labeled one.
• Multi-task pipelines where unlabeled data comes from different tasks and its distribution is unknown.
• Any long-tailed recognition setting where labeled samples are scarce and enumeration of the class distribution is impractical.
• Deployments where the labeled distribution is inverse to the unlabeled one — a case explicitly tested in the inverse setting.

Industry relevance: The method is built on top of an existing SSL algorithm (FixMatch) and evaluated on a standard codebase (USB), so it is a drop-in style modification — an extra classifier plus bias correction — rather than a new training paradigm. Reported gains on ImageNet-127 (62.3 and 69.4 versus 35.7 and 44.0 for FixMatch) suggest the approach scales beyond small CIFAR-style benchmarks.

Future Directions

• Finer-grained class grouping: The paper asks explicitly why classes are split only into head and non-head, and reports Table 4 showing 2/3/4 partitions produce similar results; whether this holds for distributions beyond those tested is left open.
• Removing the anchor-distribution dependence: The method still uses 5 predefined anchors (consist, uniform, inverse, Gaussian, inverse-Gaussian) and sets c per anchor. A fully anchor-free variant is not demonstrated.
• Robustness of the bias proxy: The approach assumes the output classifier's bias term captures optimization imbalance cleanly. The paper does not report an analysis of when this assumption fails, such as under very chaotic or wholly unknown distributions.
• Beyond image classification: All reported experiments are on CIFAR10-LT, CIFAR100-LT, STL10-LT, and ImageNet-127; no other modalities or tasks are evaluated.

Target Audience

Researchers and practitioners working on semi-supervised learning, long-tailed recognition, and class-imbalanced training who are already familiar with pseudo-labeling methods such as FixMatch and with dual-classifier CISSL baselines (ABC, DASO, ACR, CPE, SimPro, CDMAD). It is also relevant to engineers deploying models where labeled data is scarce and the unlabeled class mix is unknown or mismatched, and who want a method that requires only an added classifier head and an inference-time bias correction.

Authors’ abstract

Class imbalance remains a critical challenge in semi-supervised learning (SSL), especially when distributional mismatches between labeled and unlabeled data lead to biased classification. Although existing methods address this issue by adjusting logits based on the estimated class distribution of unlabeled data, they often handle model imbalance in a coarse-grained manner, conflating data imbalance with bias arising from varying class-specific learning difficulties. To address this issue, we propose a unified framework, SC-SSL, which suppresses model bias through decoupled sampling control. During training, we identify the key variables for sampling control under ideal conditions. By introducing a classifier with explicit expansion capability and adaptively adjusting sampling probabilities across different data distributions, SC-SSL mitigates feature-level imbalance for minority classes. In the inference phase, we further analyze the weight imbalance of the linear classifier and apply post-hoc sampling control with an optimization bias vector to directly calibrate the logits. Extensive experiments across various benchmark datasets and distribution settings validate the consistency and state-of-the-art performance of SC-SSL.