# Dependent Multinomial Models Made Easy: Stick Breaking with the P'olya-Gamma Augmentation

arXiv: https://arxiv.org/abs/1506.05843 · HTML: https://mlsanity.com/paper/1506.05843/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2015.

## Built on by (most cited first, 26 of 26)

- [Personalized Federated Learning with Gaussian Processes](https://mlsanity.com/paper/2106.15482/) (2021)
- [GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning](https://mlsanity.com/paper/2102.07868/) (2021)
- [Scalable Cross Validation Losses for Gaussian Process Models](https://mlsanity.com/paper/2105.11535/) (2021)
- [Guided Deep Kernel Learning](https://mlsanity.com/paper/2302.09574/) (2023)
- [HiGen: Hierarchical Graph Generative Networks](https://mlsanity.com/paper/2305.19337/) (2023)
- [Bayesian Non-linear Latent Variable Modeling via Random Fourier Features](https://mlsanity.com/paper/2306.08352/) (2023)
- [Gradient-free variational learning with conditional mixture networks](https://mlsanity.com/paper/2408.16429/) (2024)
- [Bayesian Principles Improve Prompt Learning In Vision-Language Models](https://mlsanity.com/paper/2504.14123/) (2025)
- [AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Models](https://mlsanity.com/paper/2505.24784/) (2025)
- [Simplex-to-Euclidean Bijections for Categorical Flow Matching](https://mlsanity.com/paper/2510.27480/) (2025)
- [Bayesian Classifier Fusion with an Explicit Model of Correlation](https://mlsanity.com/paper/2106.01770/) (2021)
- [Easy Variational Inference for Categorical Models via an Independent Binary Approximation](https://mlsanity.com/paper/2206.00093/) (2022)
- [Rethinking Approximate Gaussian Inference in Classification](https://mlsanity.com/paper/2502.03366/) (2025)
- [Hyperbolic Secant representation of the logistic function: Application to probabilistic Multiple Instance Learning for CT intracranial hemorrhage detection](https://mlsanity.com/paper/2403.14829/) (2024)
- [Bayesian Inference for Gamma Models](https://mlsanity.com/paper/2106.01906/) (2021)
- [Recurrent Coupled Topic Modeling over Sequential Documents](https://mlsanity.com/paper/2106.13732/) (2021)
- [On Hierarchical Multi-Resolution Graph Generative Models](https://mlsanity.com/paper/2303.03293/) (2023)
- [On the Calibration and Uncertainty with P'olya-Gamma Augmentation for Dialog Retrieval Models](https://mlsanity.com/paper/2303.08606/) (2023)
- [Video-based Surgical Skill Assessment using Tree-based Gaussian Process Classifier](https://mlsanity.com/paper/2312.10208/) (2023)
- [Scalable Inference for Bayesian Multinomial Logistic-Normal Dynamic Linear Models](https://mlsanity.com/paper/2410.05548/) (2024)
- [Bayesian Inference in Recurrent Explicit Duration Switching Linear Dynamical Systems](https://mlsanity.com/paper/2411.04280/) (2024)
- [Effects of Structural Allocation of Geometric Task Diversity in Linear Meta-Learning Models](https://mlsanity.com/paper/2509.18349/) (2025)
- [Horseshoe Mixtures-of-Experts (HS-MoE)](https://mlsanity.com/paper/2601.09043/) (2026)
- [Fast Compute for ML Optimization](https://mlsanity.com/paper/2602.14280/) (2026)
- [Video-based Surgical Skill Assessment Using Dynamics-and-Uncertainty-Aware Tree-based Gaussian Process Classifier](https://mlsanity.com/paper/2609.24619/) (2026)
- [Variational Filtering for Bayesian Last Layers with an Independent Binary Probit Surrogate](https://mlsanity.com/paper/2610.16578/) (2026)

## Abstract (arXiv)

Many practical modeling problems involve discrete data that are best represented as draws from multinomial or categorical distributions. For example, nucleotides in a DNA sequence, children's names in a given state and year, and text documents are all commonly modeled with multinomial distributions. In all of these cases, we expect some form of dependency between the draws: the nucleotide at one position in the DNA strand may depend on the preceding nucleotides, children's names are highly correlated from year to year, and topics in text may be correlated and dynamic. These dependencies are not naturally captured by the typical Dirichlet-multinomial formulation. Here, we leverage a logistic stick-breaking representation and recent innovations in P'olya-gamma augmentation to reformulate the multinomial distribution in terms of latent variables with jointly Gaussian likelihoods, enabling us to take advantage of a host of Bayesian inference techniques for Gaussian models with minimal overhead.