# Regularized EM algorithm

arXiv: https://arxiv.org/abs/2303.14989 · HTML: https://mlsanity.com/paper/2303.14989/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 17 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 6 of 6)

- [Regularized EM Algorithms: A Unified Framework and Statistical Guarantees](https://mlsanity.com/paper/1511.08551/) (2015)
- [Regularized Tyler's Scatter Estimator: Existence, Uniqueness, and Algorithms](https://mlsanity.com/paper/1407.3079/) (2014)
- [A flexible EM-like clustering algorithm for noisy data](https://mlsanity.com/paper/1907.01660/) (2019)
- [Shrinking the eigenvalues of M-estimators of covariance matrix](https://mlsanity.com/paper/2006.10005/) (2020)
- [Robust classification with flexible discriminant analysis in heterogeneous data](https://mlsanity.com/paper/2201.02967/) (2022)
- [Optimal shrinkage covariance matrix estimation under random sampling from elliptical distributions](https://mlsanity.com/paper/1808.10188/) (2018)

## Abstract (arXiv)

Expectation-Maximization (EM) algorithm is a widely used iterative algorithm for computing (local) maximum likelihood estimate (MLE). It can be used in an extensive range of problems, including the clustering of data based on the Gaussian mixture model (GMM). Numerical instability and convergence problems may arise in situations where the sample size is not much larger than the data dimensionality. In such low sample support (LSS) settings, the covariance matrix update in the EM-GMM algorithm may become singular or poorly conditioned, causing the algorithm to crash. On the other hand, in many signal processing problems, a priori information can be available indicating certain structures for different cluster covariance matrices. In this paper, we present a regularized EM algorithm for GMM-s that can make efficient use of such prior knowledge as well as cope with LSS situations. The method aims to maximize a penalized GMM likelihood where regularized estimation may be used to ensure positive definiteness of covariance matrix updates and shrink the estimators towards some structured target covariance matrices. We show that the theoretical guarantees of convergence hold, leading to better performing EM algorithm for structured covariance matrix models or with low sample settings.