# Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysis

arXiv: https://arxiv.org/abs/2311.01052 · HTML: https://mlsanity.com/paper/2311.01052/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 2 Nov 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networks](https://mlsanity.com/paper/1511.06314/) (2015)
- [Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow](https://mlsanity.com/paper/1802.07095/) (2018)
- [Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses](https://mlsanity.com/paper/1612.00197/) (2016)
- [Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction](https://mlsanity.com/paper/1906.03631/) (2019)
- [Distribution Calibration for Regression](https://mlsanity.com/paper/1905.06023/) (2019)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles](https://mlsanity.com/paper/1606.07839/) (2016)
- [Improving Regression Performance with Distributional Losses](https://mlsanity.com/paper/1806.04613/) (2018)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Large Loss Matters in Weakly Supervised Multi-Label Classification](https://mlsanity.com/paper/2206.03740/) (2022)
- [DMCL: Distillation Multiple Choice Learning for Multimodal Action Recognition](https://mlsanity.com/paper/1912.10982/) (2019)
- [Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network](https://mlsanity.com/paper/1710.10059/) (2017)
- [Confident Multiple Choice Learning](https://mlsanity.com/paper/1706.03475/) (2017)
- [A multi-room reverberant dataset for sound event localization and detection](https://mlsanity.com/paper/1905.08546/) (2019)
- [Deep Learning based Multi-Source Localization with Source Splitting and its Effectiveness in Multi-Talker Speech Recognition](https://mlsanity.com/paper/2102.07955/) (2021)
- [DiverseNet: When One Right Answer is not Enough](https://mlsanity.com/paper/2008.10634/) (2020)
- [PILOT: Introducing Transformers for Probabilistic Sound Event Localization](https://mlsanity.com/paper/2106.03903/) (2021)
- [Exploiting Attention-based Sequence-to-Sequence Architectures for Sound Event Localization](https://mlsanity.com/paper/2103.00417/) (2021)
- [DsMCL: Dual-Level Stochastic Multiple Choice Learning for Multi-Modal Trajectory Prediction](https://mlsanity.com/paper/2003.08638/) (2020)

## Built on by (most cited first, 12 of 12)

- [Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing](https://mlsanity.com/paper/2407.15580/) (2024)
- [From Samples to Scenarios: A New Paradigm for Probabilistic Forecasting](https://mlsanity.com/paper/2509.19975/) (2025)
- [ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation](https://mlsanity.com/paper/2312.06386/) (2023)
- [Winner-takes-all learners are geometry-aware conditional density estimators](https://mlsanity.com/paper/2406.04706/) (2024)
- [MATPAC++: Enhanced Masked Latent Prediction for Self-Supervised Audio Representation Learning](https://mlsanity.com/paper/2508.12709/) (2025)
- [Hierarchical Uncertainty Exploration via Feedforward Posterior Trees](https://mlsanity.com/paper/2405.15719/) (2024)
- [CryoSPIN: Improving Ab-Initio Cryo-EM Reconstruction with Semi-Amortized Pose Inference](https://mlsanity.com/paper/2406.10455/) (2024)
- [Winner-takes-all for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2506.05515/) (2025)
- [Multiple Choice Learning for Efficient Speech Separation with Many Speakers](https://mlsanity.com/paper/2411.18497/) (2024)
- [Multiple Choice Learning of Low-Rank Adapters for Language Modeling](https://mlsanity.com/paper/2507.10419/) (2025)
- [Conditional Distribution Quantization in Machine Learning](https://mlsanity.com/paper/2502.07151/) (2025)
- [SHAMaNS: Sound Localization with Hybrid Alpha-Stable Spatial Measure and Neural Steerer](https://mlsanity.com/paper/2506.18954/) (2025)

## Abstract (arXiv)

We introduce Resilient Multiple Choice Learning (rMCL), an extension of the MCL approach for conditional distribution estimation in regression settings where multiple targets may be sampled for each training input. Multiple Choice Learning is a simple framework to tackle multimodal density estimation, using the Winner-Takes-All (WTA) loss for a set of hypotheses. In regression settings, the existing MCL variants focus on merging the hypotheses, thereby eventually sacrificing the diversity of the predictions. In contrast, our method relies on a novel learned scoring scheme underpinned by a mathematical framework based on Voronoi tessellations of the output space, from which we can derive a probabilistic interpretation. After empirically validating rMCL with experiments on synthetic data, we further assess its merits on the sound source localization problem, demonstrating its practical usefulness and the relevance of its interpretation.