# Winner-takes-all learners are geometry-aware conditional density estimators

arXiv: https://arxiv.org/abs/2406.04706 · HTML: https://mlsanity.com/paper/2406.04706/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 63 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks](https://mlsanity.com/paper/1809.10732/) (2018)
- [On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks](https://mlsanity.com/paper/2203.09168/) (2022)
- [Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses](https://mlsanity.com/paper/1612.00197/) (2016)
- [Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction](https://mlsanity.com/paper/1906.03631/) (2019)
- [Generating Multiple Hypotheses for 3D Human Pose Estimation with Mixture Density Network](https://mlsanity.com/paper/1904.05547/) (2019)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles](https://mlsanity.com/paper/1606.07839/) (2016)
- [Improving Regression Performance with Distributional Losses](https://mlsanity.com/paper/1806.04613/) (2018)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction](https://mlsanity.com/paper/2104.08277/) (2021)
- [DMCL: Distillation Multiple Choice Learning for Multimodal Action Recognition](https://mlsanity.com/paper/1912.10982/) (2019)
- [Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network](https://mlsanity.com/paper/1710.10059/) (2017)
- [Confident Multiple Choice Learning](https://mlsanity.com/paper/1706.03475/) (2017)
- [Structural Consistency and Controllability for Diverse Colorization](https://mlsanity.com/paper/1809.02129/) (2018)
- [Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysis](https://mlsanity.com/paper/2311.01052/) (2023)
- [Localization, Detection and Tracking of Multiple Moving Sound Sources with a Convolutional Recurrent Neural Network](https://mlsanity.com/paper/1904.12769/) (2019)
- [Theoretical Analysis of the k-Means Algorithm - A Survey](https://mlsanity.com/paper/1602.08254/) (2016)
- [PILOT: Introducing Transformers for Probabilistic Sound Event Localization](https://mlsanity.com/paper/2106.03903/) (2021)
- [Voronoi Density Estimator for High-Dimensional Data: Computation, Compactification and Convergence](https://mlsanity.com/paper/2206.08051/) (2022)
- [Exploiting Attention-based Sequence-to-Sequence Architectures for Sound Event Localization](https://mlsanity.com/paper/2103.00417/) (2021)
- [An Efficient and Continuous Voronoi Density Estimator](https://mlsanity.com/paper/2210.03964/) (2022)

## Built on by (most cited first, 6 of 6)

- [Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealing](https://mlsanity.com/paper/2407.15580/) (2024)
- [From Samples to Scenarios: A New Paradigm for Probabilistic Forecasting](https://mlsanity.com/paper/2509.19975/) (2025)
- [Winner-takes-all for Multivariate Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2506.05515/) (2025)
- [TimePre: Bridging Accuracy, Efficiency, and Stability in Probabilistic Time-Series Forecasting](https://mlsanity.com/paper/2511.18539/) (2025)
- [Multiple Choice Learning of Low-Rank Adapters for Language Modeling](https://mlsanity.com/paper/2507.10419/) (2025)
- [Conditional Distribution Quantization in Machine Learning](https://mlsanity.com/paper/2502.07151/) (2025)

## Abstract (arXiv)

Winner-takes-all training is a simple learning paradigm, which handles ambiguous tasks by predicting a set of plausible hypotheses. Recently, a connection was established between Winner-takes-all training and centroidal Voronoi tessellations, showing that, once trained, hypotheses should quantize optimally the shape of the conditional distribution to predict. However, the best use of these hypotheses for uncertainty quantification is still an open question. In this work, we show how to leverage the appealing geometric properties of the Winner-takes-all learners for conditional density estimation, without modifying its original training scheme. We theoretically establish the advantages of our novel estimator both in terms of quantization and density estimation, and we demonstrate its competitiveness on synthetic and real-world datasets, including audio data.