# Confident Multiple Choice Learning

arXiv: https://arxiv.org/abs/1706.03475 · HTML: https://mlsanity.com/paper/1706.03475/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2017.

## Built on by (most cited first, 24 of 24)

- [SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustness](https://mlsanity.com/paper/2111.09277/) (2021)
- [Residual Mixture of Experts](https://mlsanity.com/paper/2204.09636/) (2022)
- [Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation](https://mlsanity.com/paper/2103.05271/) (2021)
- [Embracing Uncertainty: Decoupling and De-bias for Robust Temporal Grounding](https://mlsanity.com/paper/2103.16848/) (2021)
- [DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples](https://mlsanity.com/paper/2110.13740/) (2021)
- [Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysis](https://mlsanity.com/paper/2311.01052/) (2023)
- [Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing](https://mlsanity.com/paper/2407.15580/) (2024)
- [MCL-GAN: Generative Adversarial Networks with Multiple Specialized Discriminators](https://mlsanity.com/paper/2107.07260/) (2021)
- [kFolden: k-Fold Ensemble for Out-Of-Distribution Detection](https://mlsanity.com/paper/2108.12731/) (2021)
- [EUCLID: Towards Efficient Unsupervised Reinforcement Learning with Multi-choice Dynamics Model](https://mlsanity.com/paper/2210.00498/) (2022)
- [Self-Activating Neural Ensembles for Continual Reinforcement Learning](https://mlsanity.com/paper/2301.00141/) (2022)
- [ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation](https://mlsanity.com/paper/2312.06386/) (2023)
- [Winner-takes-all learners are geometry-aware conditional density estimators](https://mlsanity.com/paper/2406.04706/) (2024)
- [Hierarchical Uncertainty Exploration via Feedforward Posterior Trees](https://mlsanity.com/paper/2405.15719/) (2024)
- [CryoSPIN: Improving Ab-Initio Cryo-EM Reconstruction with Semi-Amortized Pose Inference](https://mlsanity.com/paper/2406.10455/) (2024)
- [Annealed Winner-Takes-All for Motion Forecasting](https://mlsanity.com/paper/2409.11172/) (2024)
- [Winner-takes-all for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2506.05515/) (2025)
- [Multiple Choice Learning of Low-Rank Adapters for Language Modeling](https://mlsanity.com/paper/2507.10419/) (2025)
- [Multiple Hypothesis Dropout: Estimating the Parameters of Multi-Modal Output Distributions](https://mlsanity.com/paper/2312.11735/) (2023)
- [Auxiliary Class Based Multiple Choice Learning](https://mlsanity.com/paper/2108.02949/) (2021)
- [Masked Vector Quantization](https://mlsanity.com/paper/2301.06626/) (2023)
- [Towards Diverse Perspective Learning with Selection over Multiple Temporal Poolings](https://mlsanity.com/paper/2403.09749/) (2024)
- [Conditional Distribution Quantization in Machine Learning](https://mlsanity.com/paper/2502.07151/) (2025)
- [CHASE: Competing Hypotheses for Ambiguity-Aware Selective Prediction](https://mlsanity.com/paper/2605.01346/) (2026)

## Abstract (arXiv)

Ensemble methods are arguably the most trustworthy techniques for boosting the performance of machine learning models. Popular independent ensembles (IE) relying on naive averaging/voting scheme have been of typical choice for most applications involving deep neural networks, but they do not consider advanced collaboration among ensemble models. In this paper, we propose new ensemble methods specialized for deep neural networks, called confident multiple choice learning (CMCL): it is a variant of multiple choice learning (MCL) via addressing its overconfidence issue.In particular, the proposed major components of CMCL beyond the original MCL scheme are (i) new loss, i.e., confident oracle loss, (ii) new architecture, i.e., feature sharing and (iii) new training method, i.e., stochastic labeling. We demonstrate the effect of CMCL via experiments on the image classification on CIFAR and SVHN, and the foreground-background segmentation on the iCoseg. In particular, CMCL using 5 residual networks provides 14.05% and 6.60% relative reductions in the top-1 error rates from the corresponding IE scheme for the classification task on CIFAR and SVHN, respectively.