# Variational Approximation Error in Bayesian Non-negative Matrix Factorization

arXiv: https://arxiv.org/abs/1809.02963 · HTML: https://mlsanity.com/paper/1809.02963/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2018.

## Built on by (most cited first, 5 of 5)

- [Bayesian Generalization Error in Linear Neural Networks with Concept Bottleneck Structure and Multitask Formulation](https://mlsanity.com/paper/2303.09154/) (2023)
- [Variational Bayesian Neural Networks via Resolution of Singularities](https://mlsanity.com/paper/2302.06035/) (2023)
- [Upper Bound of Bayesian Generalization Error in Partial Concept Bottleneck Model (CBM): Partial CBM outperforms naive CBM](https://mlsanity.com/paper/2403.09206/) (2024)
- [An Asymptotic Equation Linking WAIC and WBIC in Singular Models](https://mlsanity.com/paper/2505.13902/) (2025)
- [Singularities of Non-negative Matrix Factorization and their application to Bayesian inference](https://mlsanity.com/paper/2609.34043/) (2026)

## Abstract (arXiv)

Non-negative matrix factorization (NMF) is a knowledge discovery method that is used in many fields. Variational inference and Gibbs sampling methods for it are also wellknown. However, the variational approximation error has not been clarified yet, because NMF is not statistically regular and the prior distribution used in variational Bayesian NMF (VBNMF) has zero or divergence points. In this paper, using algebraic geometrical methods, we theoretically analyze the difference in negative log evidence (a.k.a. free energy) between VBNMF and Bayesian NMF, i.e., the Kullback-Leibler divergence between the variational posterior and the true posterior. We derive an upper bound for the learning coefficient (a.k.a. the real log canonical threshold) in Bayesian NMF. By using the upper bound, we find a lower bound for the approximation error, asymptotically. The result quantitatively shows how well the VBNMF algorithm can approximate Bayesian NMF; the lower bound depends on the hyperparameters and the true nonnegative rank. A numerical experiment demonstrates the theoretical result.