# A derivation of variational message passing (VMP) for latent Dirichlet allocation (LDA)

arXiv: https://arxiv.org/abs/2111.01480 · HTML: https://mlsanity.com/paper/2111.01480/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 48 references that are in the map.
- First posted on arXiv 2 Nov 2021.

## Builds on (most cited first, 6 of 6)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Latent Dirichlet Allocation (LDA) and Topic modeling: models, applications, a survey](https://mlsanity.com/paper/1711.04305/) (2017)
- [Structured Stochastic Variational Inference](https://mlsanity.com/paper/1404.4114/) (2014)
- [Strengthening Probabilistic Graphical Models: The Purge-and-merge Algorithm](https://mlsanity.com/paper/2110.00091/) (2021)
- [ALBU: An approximate Loopy Belief message passing algorithm for LDA to improve performance on small data sets](https://mlsanity.com/paper/2110.00635/) (2021)

## Abstract (arXiv)

Latent Dirichlet Allocation (LDA) is a probabilistic model used to uncover latent topics in a corpus of documents. Inference is often performed using variational Bayes (VB) algorithms, which calculate a lower bound to the posterior distribution over the parameters. Deriving the variational update equations for new models requires considerable manual effort; variational message passing (VMP) has emerged as a "black-box" tool to expedite the process of variational inference. But applying VMP in practice still presents subtle challenges, and the existing literature does not contain the steps that are necessary to implement VMP for the standard smoothed LDA model, nor are available black-box probabilistic graphical modelling software able to do the word-topic updates necessary to implement LDA. In this paper, we therefore present a detailed derivation of the VMP update equations for LDA. We see this as a first step to enabling other researchers to calculate the VMP updates for similar graphical models.