# Damped Anderson acceleration with restarts and monotonicity control for accelerating EM and EM-like algorithms

arXiv: https://arxiv.org/abs/1803.06673 · HTML: https://mlsanity.com/paper/1803.06673/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2018.

## Built on by (most cited first, 4 of 4)

- [Stochastic Anderson Mixing for Nonconvex Stochastic Optimization](https://mlsanity.com/paper/2110.01543/) (2021)
- [Non-negative matrix factorization algorithms generally improve topic model fits](https://mlsanity.com/paper/2105.13440/) (2021)
- [Anderson Acceleration as a Krylov Method with Application to Asymptotic Convergence Analysis](https://mlsanity.com/paper/2109.14181/) (2021)
- [AMICA-Python: Adaptive Mixture Independent Component Analysis with Anderson Acceleration](https://mlsanity.com/paper/2607.18568/) (2026)

## Abstract (arXiv)

The expectation-maximization (EM) algorithm is a well-known iterative method for computing maximum likelihood estimates from incomplete data. Despite its numerous advantages, a main drawback of the EM algorithm is its frequently observed slow convergence which often hinders the application of EM algorithms in high-dimensional problems or in other complex settings.To address the need for more rapidly convergent EM algorithms, we describe a new class of acceleration schemes that build on the Anderson acceleration technique for speeding fixed-point iterations. Our approach is effective at greatly accelerating the convergence of EM algorithms and is automatically scalable to high dimensional settings. Through the introduction of periodic algorithm restarts and a damping factor, our acceleration scheme provides faster and more robust convergence when compared to un-modified Anderson acceleration while also improving global convergence. Crucially, our method works as an "off-the-shelf" method in that it may be directly used to accelerate any EM algorithm without relying on the use of any model-specific features or insights. Through a series of simulation studies involving five representative problems, we show that our algorithm is substantially faster than the existing state-of-art acceleration schemes.