# Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences

arXiv: https://arxiv.org/abs/1609.00978 · HTML: https://mlsanity.com/paper/1609.00978/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2016.

## Built on by (most cited first, 30 of 34)

- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow](https://mlsanity.com/paper/2301.01766/) (2023)
- [Offline Reinforcement Learning with Closed-Form Policy Improvement Operators](https://mlsanity.com/paper/2211.15956/) (2022)
- [A theoretical perspective on mode collapse in variational inference](https://mlsanity.com/paper/2410.13300/) (2024)
- [Breaking the gridlock in Mixture-of-Experts: Consistent and Efficient Algorithms](https://mlsanity.com/paper/1802.07417/) (2018)
- [Improved Convergence Guarantees for Learning Gaussian Mixture Models by EM and Gradient EM](https://mlsanity.com/paper/2101.00575/) (2021)
- [Toward Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixture Models](https://mlsanity.com/paper/2407.00490/) (2024)
- [Optimal Clustering by Lloyd Algorithm for Low-Rank Mixture Model](https://mlsanity.com/paper/2207.04600/) (2022)
- [A sampling-based approach for efficient clustering in large datasets](https://mlsanity.com/paper/2112.14793/) (2021)
- [Soft Mixture Denoising: Beyond the Expressive Bottleneck of Diffusion Models](https://mlsanity.com/paper/2309.14068/) (2023)
- [Blind Channel Estimation and Joint Symbol Detection with Data-Driven Factor Graphs](https://mlsanity.com/paper/2401.12627/) (2024)
- [Nonparametric MLE for Gaussian Location Mixtures: Certified Computation and Generic Behavior](https://mlsanity.com/paper/2503.20193/) (2025)
- [Transformers as Unsupervised Learning Algorithms: A study on Gaussian Mixtures](https://mlsanity.com/paper/2505.11918/) (2025)
- [Convergence Dynamics of Over-Parameterized Score Matching for a Single Gaussian](https://mlsanity.com/paper/2511.22069/) (2025)
- [A Geometric Approach to k-means](https://mlsanity.com/paper/2201.04822/) (2022)
- [Beyond EM Algorithm on Over-specified Two-Component Location-Scale Gaussian Mixtures](https://mlsanity.com/paper/2205.11078/) (2022)
- [Likelihood Adjusted Semidefinite Programs for Clustering Heterogeneous Data](https://mlsanity.com/paper/2209.15097/) (2022)
- [Big Learning Expectation Maximization](https://mlsanity.com/paper/2312.11926/) (2023)
- [Conformal Prediction for Generative Models via Adaptive Cluster-Based Density Estimation](https://mlsanity.com/paper/2601.22298/) (2026)
- [On the Optimization Landscape of Maximum Mean Discrepancy](https://mlsanity.com/paper/2110.13452/) (2021)
- [Fast Learning of MNL Model from General Partial Rankings with Application to Network Formation Modeling](https://mlsanity.com/paper/2112.15575/) (2021)
- [LapGM: A Multisequence MR Bias Correction and Normalization Model](https://mlsanity.com/paper/2209.13619/) (2022)
- [An Efficient 1 Iteration Learning Algorithm for Gaussian Mixture Model And Gaussian Mixture Embedding For Neural Network](https://mlsanity.com/paper/2308.09444/) (2023)
- [Joint Problems in Learning Multiple Dynamical Systems](https://mlsanity.com/paper/2311.02181/) (2023)
- [Diffusion Model Conditioning on Gaussian Mixture Model and Negative Gaussian Mixture Gradient](https://mlsanity.com/paper/2401.11261/) (2024)
- [In-context Learning for Mixture of Linear Regressions: Existence, Generalization and Training Dynamics](https://mlsanity.com/paper/2410.14183/) (2024)
- [Information Theoretic Structured Generative Modeling](https://mlsanity.com/paper/2110.05794/) (2021)
- [Mean Estimation in High-Dimensional Binary Markov Gaussian Mixture Models](https://mlsanity.com/paper/2206.02455/) (2022)
- [Masked Part-Of-Speech Model: Does Modeling Long Context Help Unsupervised POS-tagging?](https://mlsanity.com/paper/2206.14969/) (2022)
- [Unsupervised spectral-band feature identification for optimal process discrimination](https://mlsanity.com/paper/2212.03800/) (2022)

## Abstract (arXiv)

We provide two fundamental results on the population (infinite-sample) likelihood function of Gaussian mixture models with M ≥ 3 components. Our first main result shows that the population likelihood function has bad local maxima even in the special case of equally-weighted mixtures of well-separated and spherical Gaussians. We prove that the log-likelihood value of these bad local maxima can be arbitrarily worse than that of any global optimum, thereby resolving an open question of Srebro (2007). Our second main result shows that the EM algorithm (or a first-order variant of it) with random initialization will converge to bad critical points with probability at least 1-e-Omega(M). We further establish that a first-order variant of EM will not converge to strict saddle points almost surely, indicating that the poor performance of the first-order method can be attributed to the existence of bad local maxima rather than bad saddle points. Overall, our results highlight the necessity of careful initialization when using the EM algorithm in practice, even when applied in highly favorable settings.