Learning Credal Ensembles via Distributionally Robust Optimization

AI summary

Overview

Research area: Uncertainty quantification in deep learning, specifically epistemic uncertainty (EU) estimation via credal sets and distributionally robust optimization (DRO).

Technical level: Intermediate. Readers should be comfortable with classification models, softmax probabilities, ensembles, and basic notions of entropy, but the paper's core idea is explained without requiring deep measure-theoretic background.

Scope: The paper proposes CreDRO, a method that builds a credal (set-valued) predictor by training ensemble members under different degrees of relaxation of the i.i.d. assumption between training and test data, and evaluates it on out-of-distribution (OOD) detection and a comparison of test accuracy and calibration.

What This Paper Is About

Credal predictors output a convex set of probabilistic predictions rather than a single probability vector, which lets them express epistemic uncertainty. The authors argue that most existing credal methods define that uncertainty as disagreement caused by random weight initializations of ensemble members, which mainly reflects sensitivity to optimization randomness rather than anything more substantive. CreDRO instead defines epistemic uncertainty as disagreement among models trained under different degrees of assumed train–test distribution shift, produced by varying a DRO hyperparameter across ensemble members.

Key Contributions

1. A reformulation of epistemic uncertainty for credal ensembles as disagreement between models trained under different degrees of relaxation of the i.i.d. assumption between training and test distributions, rather than disagreement from random initializations alone.

2. CreDRO, a training scheme based on adversarially reweighted learning within the group DRO family, which instantiates the uncertainty set as a conditional value at risk (CVaR) set at level δ. The inner maximization has a closed-form solution: weight δ⁻¹ on the top-⌊δN⌋ highest-loss samples and zero elsewhere, approximated batch-wise by backpropagating only the top-δ fraction of losses.

3. A per-member hyperparameter assignment in which member i uses δᵢ = (1−δ_G)/(M−1)·(i−1) + δ_G, a uniform interpolation over [δ_G, 1] controlled by a single global hyperparameter δ_G ∈ [0.5, 1), so that ensemble members have formally distinct robustness profiles.

4. Inference that converts individual softmax vectors into class-wise probability intervals and a box credal set, with EU computed as the difference between upper and lower Shannon entropy; the paper also compares the box credal set against a convex-hull alternative.

Main Findings

• OOD detection on CIFAR10 as in-distribution data (M = 20, average of 3 runs): CreDRO achieves the best AUROC in every column of Table 1 — 97.4 ± 0.1 on SVHN, 92.7 ± 0.1 on Places365, 92.5 ± 0.1 on CIFAR100, 96.4 ± 0.0 on FMNIST, and 91.1 ± 0.1 on ImageNet. For comparison, the standard deep ensemble (DE) scores 94.8 ± 0.3, 90.0 ± 0.2, 90.6 ± 0.0, 92.9 ± 0.3, and 88.9 ± 0.1 on the same five datasets, and CreDE scores 94.3 ± 0.3, 91.8 ± 0.0, 91.2 ± 0.0, 95.1 ± 0.2, and 88.4 ± 0.1.

• DRO matters beyond credal wrapping: EN-DRO, which uses the same DRO training framework but does not produce credal predictions, scores 95.7 ± 0.0 (SVHN), 91.1 ± 0.1 (Places365), 91.6 ± 0.1 (CIFAR100), 94.0 ± 0.1 (FMNIST), and 90.0 ± 0.1 (ImageNet), below CreDRO on all five benchmarks.

• Point prediction accuracy and calibration improve too: Using the averaged probability vector as a single prediction on the CIFAR10 test set, CreDRO reaches test accuracy 0.9637 ± 0.0004 and ECE 0.0038 ± 0.0008, versus DE at 0.9569 ± 0.0004 accuracy and 0.0051 ± 0.0004 ECE (Table 2).

• Runtime is comparable: At M = 5 on a single Nvidia A100-SXM4-40GB GPU, CreDRO takes 6567.93 ± 35.78 seconds to train, 1.89 ± 0.02 seconds for inference, and 116.37 ± 0.30 seconds for UQ on 1000 CIFAR10 test samples. CreDE takes 6760.24 ± 52.81 seconds to train, 2.03 ± 0.02 seconds for inference, and 165.20 ± 0.96 seconds for UQ. CreEns has the heaviest UQ runtime at 308.00 ± 7.93 seconds, which the authors attribute to its convex-hull credal representation.

• Performance grows with ensemble size: Testing M ∈ {5, 10, 15} on the OOD benchmarks, the authors report that CreDRO consistently outperforms all baselines and that performance improves with larger ensemble sizes (full scores in Table 10 of the appendix, not included in the provided content).

• Robustness to the global hyperparameter: With M = 5, varying δ_G over {0.5, 0.6, 0.7, 0.8, 0.9} produces nearly flat AUROC. For example, on SVHN the scores run 96.6 ± 0.8, 96.2 ± 0.0, 95.8 ± 0.3, 96.3 ± 0.4, and 96.1 ± 0.4 respectively, and on FMNIST 95.1 ± 0.1, 95.1 ± 0.3, 94.9 ± 0.0, 95.1 ± 0.1, and 95.1 ± 0.3. The authors explain this by noting that δ_G captures only one subjective belief about worst-case divergence, while CreDRO spreads members across multiple sensitivity levels.

• Box credal sets beat convex-hull credal sets: Across M ∈ {5, 10, 15, 20}, the box credal set K_B outperforms the convex-hull set K_C on every OOD benchmark in Table 5. At M = 20, K_B scores 97.4 ± 0.1 (SVHN), 92.7 ± 0.1 (Places365), 92.5 ± 0.1 (CIFAR100), 96.4 ± 0.0 (FMNIST), and 91.1 ± 0.1 (ImageNet), while K_C scores 97.2 ± 0.1, 92.4 ± 0.1, 92.3 ± 0.1, 96.0 ± 0.1, and 90.8 ± 0.2.

• Kernel density evidence: Figure 3 reports kernel density estimates of EU for ID and OOD samples at M = 5 (first-run results), which the authors say show notably higher EU values for OOD samples than for ID samples.

• Results not reported in the provided content: The abstract mentions selective classification in medical settings, and Section 4.5 is titled "Robustness to Label Noise," but the text cuts off at the setup for that section. No medical selective classification results and no label noise results appear in the provided content, and the appendix tables referenced (Tables 8, 10, 15, and the additional figures) are not included.

Methodology in Plain English

The method starts from a standard problem: models trained by minimizing average loss assume test data look like training data, which often fails. Distributionally robust optimization instead minimizes the worst-case loss over a set of plausible distributions. The authors use a specific, tractable version of this called adversarially reweighted learning, where an adversary upweights the hardest training examples.

Concretely, they pick the top fraction δ of examples in each batch that currently have the highest loss and train only on those, discarding the rest for that member's update. A small δ means training only on the hardest examples, which approximates a strong assumed shift; a δ close to 1 means training on almost everything, which approximates the ordinary empirical risk objective.

The key idea is that no single δ is correct, so they train a whole ensemble with a spread of δ values. A single global hyperparameter δ_G sets the most aggressive member, and the remaining members are placed at even intervals between δ_G and 1. Because members are shaped by different assumptions about how much the test distribution may differ, their disagreement carries information about possible distribution shift rather than just about random seed.

At test time, the ensemble's softmax outputs are not averaged for uncertainty purposes. Instead, for each class the minimum and maximum predicted probability across members are taken, forming an interval. The collection of probability vectors consistent with all these intervals is a box credal set. Epistemic uncertainty is the gap between the highest and lowest Shannon entropy achievable inside that set, computed with SciPy. A high gap means the ensemble is genuinely conflicted; a low gap means it is not.

Why This Matters

Impact on research: The paper challenges a common design pattern in credal and ensemble-based uncertainty quantification, where diversity is obtained by changing random seeds. It shows that diversity can instead be induced by varying a formal robustness parameter, and that this yields better OOD detection than the deep ensemble baseline and several credal competitors. It also reports that no architectural modification is needed relative to a standard network, in contrast to CreDE, which doubles the number of output neurons to predict lower and upper bounds. The authors further argue that CreDRO avoids CreDE's limitation of requiring one-hot label data when using cross-entropy loss.

Real-world applications:

• Detecting inputs that fall outside a model's training distribution in deployed vision systems, evaluated here on SVHN, Places365, CIFAR100, FMNIST, and ImageNet as OOD sources for a CIFAR10 model.
• Selective prediction and learning to reject or defer, where a model's confidence is used to decide whether to abstain.
• Safety-critical decision-making where the distinction between aleatoric and epistemic uncertainty changes the appropriate response.
• Bayesian optimization and other sequential decision pipelines where overconfident predictions are costly.

Industry relevance: The runtime measurements matter for adoption. CreDRO trains in 6567.93 ± 35.78 seconds at M = 5 on a single A100, slightly less than CreDE's 6760.24 ± 52.81 seconds, and its UQ time of 116.37 ± 0.30 seconds is well below CreEns at 308.00 ± 7.93 seconds. The extra cost relative to classical ensembles comes from sorting per-sample losses within each batch. The code is available at the GitHub repository listed in the paper.

Future Directions

• Extending calibration metrics to credal predictions. The authors note that ECE is defined for single-probability predictions and that a principled extension to credal sets requires further investigation.
• Producing a single representative probability vector from a credal set. The authors state this remains an open problem and is outside the scope of the work.
• Testing the approach on non-one-hot label settings and on the robustness-to-label-noise experiments described in Section 4.5, whose results are not included in the provided content.
• Applying the framework to selective classification in medical settings, as mentioned in the abstract, and exploring alternative uncertainty measures for credal sets beyond upper-minus-lower Shannon entropy, which the authors describe as an active research area.

Target Audience

Researchers and graduate students working on uncertainty quantification, trustworthy machine learning, and distributionally robust optimization, along with practitioners in safety-critical domains such as medical imaging and autonomous systems who need epistemic uncertainty estimates they can act on. Readers familiar with deep ensembles, softmax-based uncertainty decomposition, and the basics of credal sets will get the most out of it, though the core argument is accessible to anyone who understands ensemble disagreement and calibration error.

Authors’ abstract

Credal predictors are models that are aware of epistemic uncertainty and produce a convex set of probabilistic predictions. They offer a principled way to quantify predictive epistemic uncertainty (EU) and have been shown to improve model robustness in various settings. However, most state-of-the-art methods mainly define EU as disagreement caused by random training initializations, which mostly reflects sensitivity to optimization randomness rather than uncertainty from deeper sources. To address this, we define EU as disagreement among models trained with varying relaxations of the i.i.d. assumption between training and test data. Based on this idea, we propose CreDRO, which learns an ensemble of plausible models through distributionally robust optimization. As a result, CreDRO captures EU not only from training randomness but also from meaningful disagreement due to potential distribution shifts between training and test data. Empirical results show that CreDRO consistently outperforms existing credal methods on tasks such as out-of-distribution detection across multiple benchmarks and selective classification in medical applications.