# On the Global Convergence of (Fast) Incremental Expectation Maximization Methods

arXiv: https://arxiv.org/abs/1910.12521 · HTML: https://mlsanity.com/paper/1910.12521/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 12 of 12)

- [A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks](https://mlsanity.com/paper/2212.00720/) (2022)
- [Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning](https://mlsanity.com/paper/2302.11147/) (2023)
- [Federated Expectation Maximization with heterogeneity mitigation and variance reduction](https://mlsanity.com/paper/2111.02083/) (2021)
- [Convergence of online k-means](https://mlsanity.com/paper/2202.10640/) (2022)
- [Stochastic Variable Metric Proximal Gradient with variance reduction for non-convex composite optimization](https://mlsanity.com/paper/2301.00631/) (2023)
- [The Perturbed Prox-Preconditioned SPIDER algorithm for EM-based large scale learning](https://mlsanity.com/paper/2105.11732/) (2021)
- [Fast convergence of the Expectation Maximization algorithm under a logarithmic Sobolev inequality](https://mlsanity.com/paper/2407.17949/) (2024)
- [A Class of Two-Timescale Stochastic EM Algorithms for Nonconvex Latent Variable Models](https://mlsanity.com/paper/2203.10186/) (2022)
- [Towards frugal unsupervised detection of subtle abnormalities in medical imaging](https://mlsanity.com/paper/2309.02458/) (2023)
- [Revisiting Incremental Stochastic Majorization-Minimization Algorithms with Applications to Mixture of Experts](https://mlsanity.com/paper/2601.19811/) (2026)
- [Fast Model Selection and Stable Optimization for Softmax-Gated Multinomial-Logistic Mixture of Experts Models](https://mlsanity.com/paper/2602.07997/) (2026)
- [A Synthesizable RTL Implementation of Predictive Coding Networks](https://mlsanity.com/paper/2603.18066/) (2026)

## Abstract (arXiv)

The EM algorithm is one of the most popular algorithm for inference in latent data models. The original formulation of the EM algorithm does not scale to large data set, because the whole data set is required at each iteration of the algorithm. To alleviate this problem, Neal and Hinton have proposed an incremental version of the EM (iEM) in which at each iteration the conditional expectation of the latent data (E-step) is updated only for a mini-batch of observations. Another approach has been proposed by Capp'e and Moulines in which the E-step is replaced by a stochastic approximation step, closely related to stochastic gradient. In this paper, we analyze incremental and stochastic version of the EM algorithm as well as the variance reduced-version of Chen et. al. in a common unifying framework. We also introduce a new version incremental version, inspired by the SAGA algorithm by Defazio et. al. We establish non-asymptotic convergence bounds for global convergence. Numerical applications are presented in this article to illustrate our findings.