# On the Role of Channel Capacity in Learning Gaussian Mixture Models

arXiv: https://arxiv.org/abs/2202.07707 · HTML: https://mlsanity.com/paper/2202.07707/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 52 references that are in the map.
- First posted on arXiv 15 Feb 2022.

## Builds on (most cited first, 11 of 11)

- [Robust Estimators in High Dimensions without the Computational Intractability](https://mlsanity.com/paper/1604.06443/) (2016)
- [Mixture Models, Robustness, and Sum of Squares Proofs](https://mlsanity.com/paper/1711.07454/) (2017)
- [List-Decodable Robust Mean Estimation and Learning Mixtures of Spherical Gaussians](https://mlsanity.com/paper/1711.07211/) (2017)
- [On Learning Mixtures of Well-Separated Gaussians](https://mlsanity.com/paper/1710.11592/) (2017)
- [Tight bounds for learning a mixture of two gaussians](https://mlsanity.com/paper/1404.4997/) (2014)
- [Learning Mixtures of Gaussians in High Dimensions](https://mlsanity.com/paper/1503.00424/) (2015)
- [Near-optimal-sample estimators for spherical Gaussian mixtures](https://mlsanity.com/paper/1402.4746/) (2014)
- [The More, the Merrier: the Blessing of Dimensionality for Learning Large Gaussian Mixtures](https://mlsanity.com/paper/1311.2891/) (2013)
- [Uniqueness of Tensor Decompositions with Applications to Polynomial Identifiability](https://mlsanity.com/paper/1304.8087/) (2013)
- [Fourier PCA and Robust Tensor Decomposition](https://mlsanity.com/paper/1306.5825/) (2013)
- [Multi-reference alignment in high dimensions: sample complexity and phase transition](https://mlsanity.com/paper/2007.11482/) (2020)

## Built on by (most cited first, 2 of 2)

- [Active clustering with bandit feedback](https://mlsanity.com/paper/2406.11485/) (2024)
- [Low-degree Lower bounds for clustering in moderate dimension](https://mlsanity.com/paper/2602.23023/) (2026)

## Abstract (arXiv)

This paper studies the sample complexity of learning the k unknown centers of a balanced Gaussian mixture model (GMM) in ℝd with spherical covariance matrix σ2I. In particular, we are interested in the following question: what is the maximal noise level σ2, for which the sample complexity is essentially the same as when estimating the centers from labeled measurements? To that end, we restrict attention to a Bayesian formulation of the problem, where the centers are uniformly distributed on the sphere √dSd-1. Our main results characterize the exact noise threshold σ2 below which the GMM learning problem, in the large system limit d,k→∞, is as easy as learning from labeled observations, and above which it is substantially harder. The threshold occurs at fraclog kd = frac12logleft( 1+frac1σ2 right), which is the capacity of the additive white Gaussian noise (AWGN) channel. Thinking of the set of k centers as a code, this noise threshold can be interpreted as the largest noise level for which the error probability of the code over the AWGN channel is small. Previous works on the GMM learning problem have identified the minimum distance between the centers as a key parameter in determining the statistical difficulty of learning the corresponding GMM. While our results are only proved for GMMs whose centers are uniformly distributed over the sphere, they hint that perhaps it is the decoding error probability associated with the center constellation as a channel code that determines the statistical difficulty of learning the corresponding GMM, rather than just the minimum distance.