# Non-negative matrix factorization algorithms generally improve topic model fits

arXiv: https://arxiv.org/abs/2105.13440 · HTML: https://mlsanity.com/paper/2105.13440/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 100 references that are in the map.
- First posted on arXiv 27 May 2021.

## Builds on (most cited first, 11 of 11)

- [Implicit Regularization in Matrix Factorization](https://mlsanity.com/paper/1705.09280/) (2017)
- [Coordinate Descent Algorithms](https://mlsanity.com/paper/1502.04759/) (2015)
- [A Continuous-Time View of Early Stopping for Least Squares](https://mlsanity.com/paper/1810.10082/) (2018)
- [Streaming Variational Bayes](https://mlsanity.com/paper/1307.6769/) (2013)
- [Singularity, Misspecification, and the Convergence Rate of EM](https://mlsanity.com/paper/1810.00828/) (2018)
- [The Why and How of Nonnegative Matrix Factorization](https://mlsanity.com/paper/1401.5226/) (2014)
- [Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent](https://mlsanity.com/paper/2011.01170/) (2020)
- [Algorithms for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence](https://mlsanity.com/paper/2010.01935/) (2020)
- [Semidefinite Programming Based Preconditioning for More Robust Near-Separable Nonnegative Matrix Factorization](https://mlsanity.com/paper/1310.2273/) (2013)
- [Accelerating Nonnegative Matrix Factorization Algorithms using Extrapolation](https://mlsanity.com/paper/1805.06604/) (2018)
- [Damped Anderson acceleration with restarts and monotonicity control for accelerating EM and EM-like algorithms](https://mlsanity.com/paper/1803.06673/) (2018)

## Built on by (most cited first, 2 of 2)

- [Clustering by Denoising: Latent plug-and-play diffusion for single-cell data](https://mlsanity.com/paper/2510.22835/) (2025)
- [Non--negative matrix factorization using the textitR package textsfnnmf](https://mlsanity.com/paper/2607.20084/) (2026)

## Abstract (arXiv)

In an effort to develop topic modeling methods that can be quickly applied to large data sets, we revisit the problem of maximum-likelihood estimation in topic models. It is known, at least informally, that maximum-likelihood estimation in topic models is closely related to non-negative matrix factorization (NMF). Yet, to our knowledge, this relationship has not been exploited previously to fit topic models. We show that recent advances in NMF optimization methods can be leveraged to fit topic models very efficiently, often resulting in much better fits and in less time than existing algorithms for topic models. We also formally make the connection between the NMF optimization problem and maximum-likelihood estimation for the topic model, and using this result we show that the expectation maximization (EM) algorithm for the topic model is essentially the same as the classic multiplicative updates for NMF (the only difference being that the operations are performed in a different order). Our methods are implemented in the R package fastTopics.