# Parsimonious Gaussian mixture models with piecewise-constant eigenvalue profiles

arXiv: https://arxiv.org/abs/2507.01542 · HTML: https://mlsanity.com/paper/2507.01542/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 82 references that are in the map.
- First posted on arXiv 2 Jul 2025.

## Builds on (most cited first, 14 of 14)

- [On GANs and GMMs](https://mlsanity.com/paper/1805.12462/) (2018)
- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [An Alternative to EM for Gaussian Mixture Models: Batch and Stochastic Riemannian Optimization](https://mlsanity.com/paper/1706.03267/) (2017)
- [Optimization on flag manifolds](https://mlsanity.com/paper/1907.00949/) (2019)
- [Model-based clustering based on sparse finite Gaussian mixtures](https://mlsanity.com/paper/1606.06828/) (2016)
- [Growing Tiny Networks: Spotting Expressivity Bottlenecks and Fixing Them Optimally](https://mlsanity.com/paper/2405.19816/) (2024)
- [The curse of isotropy: from principal components to principal subspaces](https://mlsanity.com/paper/2307.15348/) (2023)
- [Model-based Clustering with Sparse Covariance Matrices](https://mlsanity.com/paper/1711.07748/) (2017)
- [Effective Learning of a GMRF Mixture Model](https://mlsanity.com/paper/2005.09030/) (2020)
- [Structured prior distributions for the covariance matrix in latent factor models](https://mlsanity.com/paper/2208.07831/) (2022)
- [Exact Dimensionality Selection for Bayesian PCA](https://mlsanity.com/paper/1703.02834/) (2017)
- [Lassoing Eigenvalues](https://mlsanity.com/paper/1805.08300/) (2018)
- [Gaussian model for closed curves](https://mlsanity.com/paper/2304.01367/) (2023)
- [Eigengap Sparsity for Covariance Parsimony](https://mlsanity.com/paper/2504.10110/) (2025)

## Built on by (most cited first, 1 of 1)

- [Differentiable Expectation-Maximisation and Applications to Gaussian Mixture Model Optimal Transport](https://mlsanity.com/paper/2509.02109/) (2025)

## Abstract (arXiv)

Gaussian mixture models (GMMs) are ubiquitous in statistical learning, particularly for unsupervised problems. While full GMMs suffer from the overparameterization of their covariance matrices in high-dimensional spaces, spherical GMMs (with isotropic covariance matrices) certainly lack flexibility to fit certain anisotropic distributions. Connecting these two extremes, we introduce a new family of parsimonious GMMs with piecewise-constant covariance eigenvalue profiles. These extend several low-rank models like the celebrated mixtures of probabilistic principal component analyzers (MPPCA), by enabling any possible sequence of eigenvalue multiplicities. If the latter are prespecified, then we can naturally derive an expectation-maximization (EM) algorithm to learn the mixture parameters. Otherwise, to address the notoriously-challenging issue of jointly learning the mixture parameters and hyperparameters, we propose a componentwise penalized EM algorithm, whose monotonicity is proven. We show the superior likelihood-parsimony tradeoffs achieved by our models on a variety of unsupervised experiments: density fitting, clustering and single-image denoising.