# An Alternative to EM for Gaussian Mixture Models: Batch and Stochastic Riemannian Optimization

arXiv: https://arxiv.org/abs/1706.03267 · HTML: https://mlsanity.com/paper/1706.03267/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2017.

## Built on by (most cited first, 20 of 20)

- [On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometry](https://mlsanity.com/paper/2106.00286/) (2021)
- [On Riemannian Stochastic Approximation Schemes with Fixed Step-Size](https://mlsanity.com/paper/2102.07586/) (2021)
- [Learning with symmetric positive definite matrices via generalized Bures-Wasserstein geometry](https://mlsanity.com/paper/2110.10464/) (2021)
- [Image Modeling with Deep Convolutional Gaussian Mixture Models](https://mlsanity.com/paper/2104.12686/) (2021)
- [A sampling-based approach for efficient clustering in large datasets](https://mlsanity.com/paper/2112.14793/) (2021)
- [Accelerated Riemannian Optimization: Handling Constraints with a Prox to Bound Geometric Penalties](https://mlsanity.com/paper/2211.14645/) (2022)
- [Gromov-Wasserstein-like Distances in the Gaussian Mixture Models Space](https://mlsanity.com/paper/2310.11256/) (2023)
- [A Riemannian Newton Trust-Region Method for Fitting Gaussian Mixture Models](https://mlsanity.com/paper/2104.14957/) (2021)
- [Manifold learning and optimization using tangent space proxies](https://mlsanity.com/paper/2501.12678/) (2025)
- [Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Size](https://mlsanity.com/paper/2501.18164/) (2025)
- [Parallelised Differentiable Straightest Geodesics for 3D Meshes](https://mlsanity.com/paper/2603.15780/) (2026)
- [Vector Transport Free Riemannian LBFGS for Optimization on Symmetric Positive Definite Matrix Manifolds](https://mlsanity.com/paper/2108.11019/) (2021)
- [Riemannian Stochastic Gradient Method for Nested Composition Optimization](https://mlsanity.com/paper/2207.09350/) (2022)
- [Rieoptax: Riemannian Optimization in JAX](https://mlsanity.com/paper/2210.04840/) (2022)
- [Moving Past Single Metrics: Exploring Short-Text Clustering Across Multiple Resolutions](https://mlsanity.com/paper/2502.17020/) (2025)
- [Parsimonious Gaussian mixture models with piecewise-constant eigenvalue profiles](https://mlsanity.com/paper/2507.01542/) (2025)
- [Atlas-based Manifold Representations for Interpretable Riemannian Machine Learning](https://mlsanity.com/paper/2510.17772/) (2025)
- [Riemannian Optimization for Variance Estimation in Linear Mixed Models](https://mlsanity.com/paper/2212.09081/) (2022)
- [Probabilistic Classification by Density Estimation Using Gaussian Mixture Model and Masked Autoregressive Flow](https://mlsanity.com/paper/2310.10843/) (2023)
- [Fair Model-based Clustering](https://mlsanity.com/paper/2602.21509/) (2026)

## Abstract (arXiv)

We consider maximum likelihood estimation for Gaussian Mixture Models (Gmms). This task is almost invariably solved (in theory and practice) via the Expectation Maximization (EM) algorithm. EM owes its success to various factors, of which is its ability to fulfill positive definiteness constraints in closed form is of key importance. We propose an alternative to EM by appealing to the rich Riemannian geometry of positive definite matrices, using which we cast Gmm parameter estimation as a Riemannian optimization problem. Surprisingly, such an out-of-the-box Riemannian formulation completely fails and proves much inferior to EM. This motivates us to take a closer look at the problem geometry, and derive a better formulation that is much more amenable to Riemannian optimization. We then develop (Riemannian) batch and stochastic gradient algorithms that outperform EM, often substantially. We provide a non-asymptotic convergence analysis for our stochastic method, which is also the first (to our knowledge) such global analysis for Riemannian stochastic gradient. Numerous empirical results are included to demonstrate the effectiveness of our methods.