# On the Connection Between Non-negative Matrix Factorization and Latent Dirichlet Allocation

arXiv: https://arxiv.org/abs/2405.20542 · HTML: https://mlsanity.com/paper/2405.20542/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 50 references that are in the map.
- First posted on arXiv 30 May 2024.

## Builds on (most cited first, 6 of 6)

- [Latent Dirichlet Allocation (LDA) and Topic modeling: models, applications, a survey](https://mlsanity.com/paper/1711.04305/) (2017)
- [Algorithms for Nonnegative Matrix Factorization with the Kullback-Leibler Divergence](https://mlsanity.com/paper/2010.01935/) (2020)
- [Blind Audio Source Separation with Minimum-Volume Beta-Divergence NMF](https://mlsanity.com/paper/1907.02404/) (2019)
- [Joint Majorization-Minimization for Nonnegative Matrix Factorization with the β-divergence](https://mlsanity.com/paper/2106.15214/) (2021)
- [Majorization-minimization for Sparse Nonnegative Matrix Factorization with the β-divergence](https://mlsanity.com/paper/2207.06316/) (2022)
- [Bayesian Allocation Model: Inference by Sequential Monte Carlo for Nonnegative Tensor Factorizations and Topic Models using Polya Urns](https://mlsanity.com/paper/1903.04478/) (2019)

## Abstract (arXiv)

Non-negative matrix factorization with the generalized Kullback-Leibler divergence (NMF) and latent Dirichlet allocation (LDA) are two popular approaches for dimensionality reduction of non-negative data. Here, we show that NMF with ℓ1 normalization constraints on the columns of both matrices of the decomposition and a Dirichlet prior on the columns of one matrix is equivalent to LDA. To show this, we demonstrate that explicitly accounting for the scaling ambiguity of NMF by adding ℓ1 normalization constraints to the optimization problem allows a joint update of both matrices in the widely used multiplicative updates (MU) algorithm. When both of the matrices are normalized, the joint MU algorithm leads to probabilistic latent semantic analysis (PLSA), which is LDA without a Dirichlet prior. Our approach of deriving joint updates for NMF also reveals that a Lasso penalty on one matrix together with an ℓ1 normalization constraint on the other matrix is insufficient to induce any sparsity.