# Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing

arXiv: https://arxiv.org/abs/2407.15580 · HTML: https://mlsanity.com/paper/2407.15580/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 75 references that are in the map.
- First posted on arXiv 22 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [Attention is All You Need in Speech Separation](https://mlsanity.com/paper/2010.13154/) (2020)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow](https://mlsanity.com/paper/1802.07095/) (2018)
- [CARD: Classification and Regression Diffusion Models](https://mlsanity.com/paper/2206.07275/) (2022)
- [Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks](https://mlsanity.com/paper/1703.06284/) (2017)
- [Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses](https://mlsanity.com/paper/1612.00197/) (2016)
- [Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction](https://mlsanity.com/paper/1906.03631/) (2019)
- [Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles](https://mlsanity.com/paper/1606.07839/) (2016)
- [Sudo rm -rf: Efficient Networks for Universal Audio Source Separation](https://mlsanity.com/paper/2007.06833/) (2020)
- [TF-GridNet: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separation](https://mlsanity.com/paper/2209.03952/) (2022)
- [Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction](https://mlsanity.com/paper/2104.08277/) (2021)
- [DMCL: Distillation Multiple Choice Learning for Multimodal Action Recognition](https://mlsanity.com/paper/1912.10982/) (2019)
- [Confident Multiple Choice Learning](https://mlsanity.com/paper/1706.03475/) (2017)
- [ESPnet-se: end-to-end speech enhancement and separation toolkit designed for asr integration](https://mlsanity.com/paper/2011.03706/) (2020)
- [Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysis](https://mlsanity.com/paper/2311.01052/) (2023)
- [DiverseNet: When One Right Answer is not Enough](https://mlsanity.com/paper/2008.10634/) (2020)
- [Towards Listening to 10 People Simultaneously: An Efficient Permutation Invariant Training of Audio Source Separation Using Sinkhorn's Algorithm](https://mlsanity.com/paper/2010.11871/) (2020)
- [Winner-takes-all learners are geometry-aware conditional density estimators](https://mlsanity.com/paper/2406.04706/) (2024)
- [Hierarchical Uncertainty Exploration via Feedforward Posterior Trees](https://mlsanity.com/paper/2405.15719/) (2024)

## Built on by (most cited first, 12 of 12)

- [From Samples to Scenarios: A New Paradigm for Probabilistic Forecasting](https://mlsanity.com/paper/2509.19975/) (2025)
- [MATPAC++: Enhanced Masked Latent Prediction for Self-Supervised Audio Representation Learning](https://mlsanity.com/paper/2508.12709/) (2025)
- [Annealed Winner-Takes-All for Motion Forecasting](https://mlsanity.com/paper/2409.11172/) (2024)
- [Winner-takes-all for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2506.05515/) (2025)
- [Multiple Choice Learning for Efficient Speech Separation with Many Speakers](https://mlsanity.com/paper/2411.18497/) (2024)
- [TimePre: Bridging Accuracy, Efficiency, and Stability in Probabilistic Time-Series Forecasting](https://mlsanity.com/paper/2511.18539/) (2025)
- [Knowing When to Quit: Probabilistic Early Exits for Speech Separation](https://mlsanity.com/paper/2507.09768/) (2025)
- [Multiple Choice Learning of Low-Rank Adapters for Language Modeling](https://mlsanity.com/paper/2507.10419/) (2025)
- [From General Vision to Reliable Traversability Estimation: Adapting Vision Foundation Models for Unstructured Outdoor Environments](https://mlsanity.com/paper/2605.29565/) (2026)
- [Conditional Distribution Quantization in Machine Learning](https://mlsanity.com/paper/2502.07151/) (2025)
- [New Wide-Net-Casting Jailbreak Attacks Risk Large Models](https://mlsanity.com/paper/2605.17128/) (2026)
- [AgForce Enables Antigen-conditioned Generative Antibody Design](https://mlsanity.com/paper/2605.21610/) (2026)

## Abstract (arXiv)

We introduce Annealed Multiple Choice Learning (aMCL) which combines simulated annealing with MCL. MCL is a learning framework handling ambiguous tasks by predicting a small set of plausible hypotheses. These hypotheses are trained using the Winner-takes-all (WTA) scheme, which promotes the diversity of the predictions. However, this scheme may converge toward an arbitrarily suboptimal local minimum, due to the greedy nature of WTA. We overcome this limitation using annealing, which enhances the exploration of the hypothesis space during training. We leverage insights from statistical physics and information theory to provide a detailed description of the model training trajectory. Additionally, we validate our algorithm by extensive experiments on synthetic datasets, on the standard UCI benchmark, and on speech separation.