mlsanityData to 7 Oct 2026

Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequences

arXiv 1609.00978 ↗First posted 4 Sep 2016stat.ML cs.LG math.OCcited by 34

Not on the map. Why?

Abstract (arXiv)

We provide two fundamental results on the population (infinite-sample) likelihood function of Gaussian mixture models with M ≥ 3 components. Our first main result shows that the population likelihood function has bad local maxima even in the special case of equally-weighted mixtures of well-separated and spherical Gaussians. We prove that the log-likelihood value of these bad local maxima can be arbitrarily worse than that of any global optimum, thereby resolving an open question of Srebro (2007). Our second main result shows that the EM algorithm (or a first-order variant of it) with random initialization will converge to bad critical points with probability at least 1-e-Omega(M). We further establish that a first-order variant of EM will not converge to strict saddle points almost surely, indicating that the poor performance of the first-order method can be attributed to the existence of bad local maxima rather than bad saddle points. Overall, our results highlight the necessity of careful initialization when using the EM algorithm in practice, even when applied in highly favorable settings.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 5; 2022: 9; 2023: 7; 2024: 5; 2025: 4; 2026: 3.

34 papers.

Sort:

By area:

  1. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  2. 2023Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flowcited by 25
  3. 2022Offline Reinforcement Learning with Closed-Form Policy Improvement Operatorscited by 14
  4. 2024A theoretical perspective on mode collapse in variational inferencecited by 14
  5. 2018Breaking the gridlock in Mixture-of-Experts: Consistent and Efficient Algorithmscited by 13
  6. 2021Improved Convergence Guarantees for Learning Gaussian Mixture Models by EM and Gradient EMcited by 11
  7. 2024Toward Global Convergence of Gradient EM for Over-Parameterized Gaussian Mixture Modelscited by 6
  8. 2022Optimal Clustering by Lloyd Algorithm for Low-Rank Mixture Modelcited by 5
  9. 2021A sampling-based approach for efficient clustering in large datasetscited by 4
  10. 2023Soft Mixture Denoising: Beyond the Expressive Bottleneck of Diffusion Modelscited by 4
  11. 2024Blind Channel Estimation and Joint Symbol Detection with Data-Driven Factor Graphscited by 4
  12. 2025Nonparametric MLE for Gaussian Location Mixtures: Certified Computation and Generic Behaviorcited by 3
22 more
  1. 2022A Geometric Approach to k-meanscited by 2
  2. 2022Beyond EM Algorithm on Over-specified Two-Component Location-Scale Gaussian Mixturescited by 2
  3. 2022Likelihood Adjusted Semidefinite Programs for Clustering Heterogeneous Datacited by 2
  4. 2023Big Learning Expectation Maximizationcited by 2
  5. 2025Transformers as Unsupervised Learning Algorithms: A study on Gaussian Mixturescited by 2
  6. 2025Convergence Dynamics of Over-Parameterized Score Matching for a Single Gaussiancited by 2
  7. 2026Conformal Prediction for Generative Models via Adaptive Cluster-Based Density Estimationcited by 2
  8. 2021Fast Learning of MNL Model from General Partial Rankings with Application to Network Formation Modelingcited by 1
  9. 2022LapGM: A Multisequence MR Bias Correction and Normalization Modelcited by 1
  10. 2023An Efficient 1 Iteration Learning Algorithm for Gaussian Mixture Model And Gaussian Mixture Embedding For Neural Networkcited by 1
  11. 2023Joint Problems in Learning Multiple Dynamical Systemscited by 1
  12. 2024Diffusion Model Conditioning on Gaussian Mixture Model and Negative Gaussian Mixture Gradientcited by 1
  13. 2024In-context Learning for Mixture of Linear Regressions: Existence, Generalization and Training Dynamicscited by 1
  14. 2021Information Theoretic Structured Generative Modelingcited by 0
  15. 2021On the Optimization Landscape of Maximum Mean Discrepancycited by 0
  16. 2022Mean Estimation in High-Dimensional Binary Markov Gaussian Mixture Modelscited by 0
  17. 2022Masked Part-Of-Speech Model: Does Modeling Long Context Help Unsupervised POS-tagging?cited by 0
  18. 2022Unsupervised spectral-band feature identification for optimal process discriminationcited by 0
  19. 2023The Parametric Stability of Well-separated Spherical Gaussian Mixturescited by 0
  20. 2025Mode Collapse of Mean-Field Variational Inferencecited by 0
  21. 2026Variational Learning of Fractional Posteriorscited by 0
  22. 2026Local linear convergence of gradient methods for overparameterized Gaussian mixturescited by 0