mlsanityData to 7 Oct 2026

Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing

arXiv 2407.15580 ↗First posted 22 Jul 2024cs.LG cs.SD eess.AScited by 12

Not on the map. Why?

Abstract (arXiv)

We introduce Annealed Multiple Choice Learning (aMCL) which combines simulated annealing with MCL. MCL is a learning framework handling ambiguous tasks by predicting a small set of plausible hypotheses. These hypotheses are trained using the Winner-takes-all (WTA) scheme, which promotes the diversity of the predictions. However, this scheme may converge toward an arbitrarily suboptimal local minimum, due to the greedy nature of WTA. We overcome this limitation using annealing, which enhances the exploration of the hypothesis space during training. We leverage insights from statistical physics and information theory to provide a detailed description of the model training trajectory. Additionally, we validate our algorithm by extensive experiments on synthetic datasets, on the standard UCI benchmark, and on speech separation.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 26 of its 75 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

26 of 75 references.

References · 26

  1. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2018SDR - half-baked or well done?cited by 420
  4. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
  5. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  6. 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 213
  7. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 211
  8. 2020Attention is All You Need in Speech Separationcited by 184
  9. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  10. 2018Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flowcited by 135
  11. 2022CARD: Classification and Regression Diffusion Modelscited by 102
  12. 2017Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networkscited by 97
14 more
  1. 2016Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypothesescited by 96
  2. 2019Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Predictioncited by 96
  3. 2016Stochastic Multiple Choice Learning for Training Diverse Deep Ensemblescited by 77
  4. 2020Sudo rm -rf: Efficient Networks for Universal Audio Source Separationcited by 50
  5. 2022TF-GridNet: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separationcited by 48
  6. 2021Divide-and-Conquer for Lane-Aware Diverse Trajectory Predictioncited by 37
  7. 2019DMCL: Distillation Multiple Choice Learning for Multimodal Action Recognitioncited by 30
  8. 2017Confident Multiple Choice Learningcited by 24
  9. 2020ESPnet-se: end-to-end speech enhancement and separation toolkit designed for asr integrationcited by 22
  10. 2023Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysiscited by 12
  11. 2020DiverseNet: When One Right Answer is not Enoughcited by 11
  12. 2020Towards Listening to 10 People Simultaneously: An Efficient Permutation Invariant Training of Audio Source Separation Using Sinkhorn's Algorithmcited by 8
  13. 2024Winner-takes-all learners are geometry-aware conditional density estimatorscited by 6
  14. 2024Hierarchical Uncertainty Exploration via Feedforward Posterior Treescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 3.

12 papers.

  1. 2025From Samples to Scenarios: A New Paradigm for Probabilistic Forecastingcited by 8
  2. 2025MATPAC++: Enhanced Masked Latent Prediction for Self-Supervised Audio Representation Learningcited by 5
  3. 2024Annealed Winner-Takes-All for Motion Forecastingcited by 4
  4. 2025Winner-takes-all for Multivariate Probabilistic Time Series Forecastingcited by 4
  5. 2024Multiple Choice Learning for Efficient Speech Separation with Many Speakerscited by 3
  6. 2025TimePre: Bridging Accuracy, Efficiency, and Stability in Probabilistic Time-Series Forecastingcited by 3
  7. 2025Knowing When to Quit: Probabilistic Early Exits for Speech Separationcited by 2
  8. 2025Multiple Choice Learning of Low-Rank Adapters for Language Modelingcited by 2
  9. 2026From General Vision to Reliable Traversability Estimation: Adapting Vision Foundation Models for Unstructured Outdoor Environmentscited by 2
  10. 2025Conditional Distribution Quantization in Machine Learningcited by 0
  11. 2026New Wide-Net-Casting Jailbreak Attacks Risk Large Modelscited by 0
  12. 2026AgForce Enables Antigen-conditioned Generative Antibody Designcited by 0