BCE3S: Binary Cross-Entropy Based Tripartite Synergistic Learning for Long-tailed Recognition

AI summary

Overview

• Research area: Long-tailed recognition (LTR) in computer vision, specifically loss-function design and representation learning for class-imbalanced datasets.
• Technical level: Advanced. The paper derives gradient update rules for classifier vectors and features, and leans on neural collapse and equiangular tight frame (ETF) theory.
• Scope in one sentence: The paper proposes BCE3S, a training scheme that replaces cross-entropy with binary cross-entropy across three cooperating learning objectives, and shows it produces more compact features, better-separated classes, and more balanced classifier vectors on four long-tailed benchmarks.

What This Paper Is About

Deep models trained on long-tailed datasets — where sample counts drop sharply from head classes to tail classes — tend to be dominated by the head classes, producing imbalanced features and imbalanced classifiers. The standard cross-entropy (CE) loss couples the metrics of all classes in the denominator of its Softmax, which the authors argue re-injects and amplifies that imbalance during feature learning. The paper's goal is to build a training framework based on binary cross-entropy (BCE), whose multiple Sigmoid terms decouple the per-class metrics, and to combine three learning objectives — feature-classifier joint learning, feature-feature contrastive learning, and classifier-classifier uniform separability learning — into a single "tripartite synergistic learning" (TSL) paradigm.

Key Contributions

1. The paper introduces the concept of tripartite synergistic learning (TSL) and instantiates it as BCE3S, a BCE-based method integrating joint learning between sample features and the classifier, contrastive learning among features, and uniform learning among classifier vectors.
2. It provides an analysis of why BCE-based uniform separability learning helps train balanced classifier vectors, and of the advantage of BCE over CE in long-tailed recognition, based on the gradient update forms of the two losses.
3. It reports extensive experiments showing that BCE3S-trained models have higher intra-class compactness and inter-class separability of sample features and a more balanced classifier separability than CE-based counterparts, achieving state-of-the-art LTR results on four long-tailed datasets.
4. The authors release code at the GitHub repository listed in the paper (https://github.com/wakinghours-github/BCE3S).

Main Findings

• BCE joint learning beats CE joint learning on overall accuracy: On CIFAR100-LT with IF = 100, using ResNet32, the CE-based joint learning baseline reaches 51.48% total accuracy, while BCE-based joint learning reaches 52.88%. The gain comes from the Medium and Few subsets at the cost of lower accuracy on the Many subset, which the authors interpret as BCE paying more attention to tail classes.
• BCE contrastive learning improves all subsets: Adding BCE-based contrastive learning on top of CE joint learning raises total accuracy from 53.87% to 54.15%, and further to 54.95% when combined with BCE joint learning.
• BCE uniform learning outperforms its CE counterpart: BCE uniform learning with BCE joint learning reaches 52.67% and 53.90% in the two reported combinations, always better than the CE-based uniform learning in the same settings.
• The full tripartite BCE model is best in the ablation: BCE3S reaches 55.99% total accuracy on CIFAR100-LT (IF = 100), versus 54.14% for the CE-based tripartite counterpart.
• Feature compactness improves and balances: The four CE-based methods show similar average intra-class compactness, all approximately 82, with visible head-to-tail imbalance. BCE joint learning reaches an average compactness of 86.02, and adding the other two BCE components raises it to 95.47, while the standard deviation across classes drops from 5.55 to 1.81.
• Feature separability improves: The highest mean inter-class separability of sample features rises from 49.71 (CE joint + CE contrastive) to 50.84 for the full BCE3S combination.
• Classifier separability becomes balanced: Average classifier-vector separability is similar across methods, close to 50.50, but differs strongly across classes. The BCE uniform learning reduces the imbalance to a standard deviation of only 0.106 for the BCE joint + BCE uniform combination.
• State-of-the-art on CIFAR10-LT and CIFAR100-LT: BCE3S achieves 90.08%, 92.55%, and 95.71% on CIFAR10-LT and 59.50%, 65.23%, and 76.13% on CIFAR100-LT across the three imbalance factors tested. The paper reports these results surpassing the previous best by 1.85%, 0.66%, and 1.09% respectively across the three settings.
• State-of-the-art on ImageNet-LT: With ResNet50, BCE3S reaches 68.14% (Many), 55.90% (Medium), 40.56% (Few), and 57.85% overall, the best overall accuracy in the comparison table (the previous best listed is ProCo at 57.80%).
• Results on iNaturalist2018: With ResNet50, BCE3S trained for 180 epochs reaches 77.16% (Many), 74.45% (Medium), 72.78% (Few), and 73.99% overall; trained for 400 epochs it reaches 79.10%, 76.08%, 74.28%, and 75.91% overall. The paper's paragraph discussing this dataset is cut off in the available content.
• t-SNE visualization: On CIFAR10-LT (IF = 100) with ResNet32, the CE-based feature clusters for classes 3 and 5 ("cat" and "dog") overlap and the tail classes (8th and 9th) spread over elongated regions; BCE-based clusters occupy relatively independent regions and tail clusters become more compact as the contrastive and uniform components are added.
• Computational cost: BCE adds at most K − 1 logarithmic operations during training, described as negligible, and incurs no additional cost at testing or inference time.

Methodology in Plain English

The framework trains a deep model with a weighted sum of three losses, all written in BCE form.

1. Joint learning between features and classifier. For each sample, the model produces a feature, and a linear classifier produces one score per class. BCE treats each class score with its own Sigmoid, so a sample's score against class k is not divided by its scores against all other classes the way Softmax does. The authors normalize every classifier vector to unit norm so that head classes cannot dominate the gradients through vector magnitude. A re-sampling parameter r, together with a value p_j drawn uniformly from (0,1), randomly selects which negative class scores are used in each step.
2. Contrastive learning among features. A non-linear projection maps each feature into a projection space. The loss pulls a sample's projection toward stored projections belonging to the same class (kept in a memory bank) and pushes it away from those of other classes, using cosine similarity scaled by a temperature factor, and again writing the objective with BCE rather than Softmax.
3. Uniform separability learning among classifier vectors. This loss operates only on the classifier vectors, maximizing the separation between each vector and all others. Because it does not depend on the sample batch, it applies the same repelling force to every classifier vector in every iteration, which is the mechanism the authors credit for balanced classifier separability.

In each training iteration, a batch of samples is passed through the model; the three losses are combined with weights λ_ss and λ_cc; and the resulting tripartite loss updates both the model and the classifier. The classifier is updated by pulling forces from same-class features, repelling forces from other-class features, and the interactive term supplied by the uniform learning. The authors compare the CE and BCE gradient forms and argue that CE couples the K imbalanced exponential inner products in the Softmax denominator, re-injecting imbalance into feature learning, while BCE's Sigmoid uses only one classifier vector per pulling or repelling term.

Evaluation uses CIFAR10-LT, CIFAR100-LT, ImageNet-LT (IF = 256), and iNaturalist2018 (IF = 500), with CIFAR imbalance factors of 100, 50, and 10. Models are trained on the imbalanced training set and evaluated on the balanced test set, reporting overall accuracy plus accuracy on Many (more than 100 samples per class), Medium (100 or fewer and 20 or more), and Few (fewer than 20). Definitions of the compactness and separability metrics, dataset details, training settings, and the parameter study are stated in the paper as being in the supplementary material, which is not included in the available content.

Why This Matters

The paper argues against an implicit assumption in much of the long-tailed recognition literature: that improving the loss through re-sampling, re-weighting, or logit adjustment is enough, while the underlying CE formulation keeps coupling imbalanced class metrics. It shows that switching the base loss to BCE, whose decoupled Sigmoid terms can be adjusted class by class, changes the properties of both the learned features and the learned classifier. It also addresses a gap the authors identify — that no published work directly focuses on learning a well-balanced classifier for LTR — by adding a classifier-only objective to the training loop.

Real-world applications:

• Medical image diagnosis, where rare diseases appear far less often than common ones and tail-class recall matters clinically.
• Species monitoring and biodiversity surveys, the setting behind iNaturalist2018, where many species have few photographs.
• Autonomous driving and robotics, where rare but safety-critical objects and situations are exactly the long-tail cases.
• Industrial defect inspection and fraud/rare-event detection, where the classes of interest are by definition the rare ones.

Industry relevance: the method is a loss-level change with no added inference cost, and the authors have released code, which lowers the barrier to adopting it in existing training pipelines for imbalanced classification. The paper also notes that BCE3S can be combined with existing re-balancing techniques such as MaxNorm and BalMS to further boost performance.

Future Directions

• Plugging other losses into the TSL framework. The authors state that losses such as CE and MSE could also be integrated within the tripartite framework, but they do not explore this; comparing alternatives systematically is a natural next step.
• Extending beyond ResNet backbones and classification. The reported training uses ResNet32 and ResNet50; the paper positions BCE for transformers (citing LiVT) as prior context, leaving transformer-based BCE3S training and dense tasks such as detection or segmentation open.
• Classifier alignment versus fixed ETF classifiers. The paper contrasts its learned, feature-aligned classifier against pre-designed ETF classifiers and learnable rotations; how these approaches compare under the tripartite objective is not settled here.
• Cost and convergence analysis. Aside from the note that BCE adds at most K − 1 logarithmic operations in training and nothing at inference, the paper does not report a formal convergence or complexity analysis, and the parameter study results for λ_ss, λ_cc, and τ are deferred to supplementary material.

Target Audience

Researchers working on long-tailed and class-imbalanced recognition, loss-function and metric-learning design, and representation learning will get the most from this paper, as will practitioners who need to improve rare-class performance in deployed classifiers. Readers should be comfortable with cross-entropy and binary cross-entropy formulations, Softmax versus Sigmoid normalization, contrastive learning with memory banks, and the neural collapse / equiangular tight frame framing of classifier behavior.

Authors’ abstract

For long-tailed recognition (LTR) tasks, high intra-class compactness and inter-class separability in both head and tail classes, as well as balanced separability among all the classifier vectors, are preferred. The existing LTR methods based on cross-entropy (CE) loss not only struggle to learn features with desirable properties but also couple imbalanced classifier vectors in the denominator of its Softmax, amplifying the imbalance effects in LTR. In this paper, for the LTR, we propose a binary cross-entropy (BCE)-based tripartite synergistic learning, termed BCE3S, which consists of three components: (1) BCE-based joint learning optimizes both the classifier and sample features, which achieves better compactness and separability among features than the CE-based joint learning, by decoupling the metrics between feature and the imbalanced classifier vectors in multiple Sigmoid; (2) BCE-based contrastive learning further improves the intra-class compactness of features; (3) BCE-based uniform learning balances the separability among classifier vectors and interactively enhances the feature properties by combining with the joint learning. The extensive experiments show that the LTR model trained by BCE3S not only achieves higher compactness and separability among sample features, but also balances the classifier's separability, achieving SOTA performance on various long-tailed datasets such as CIFAR10-LT, CIFAR100-LT, ImageNet-LT, and iNaturalist2018.