# Learning noisy-OR Bayesian Networks with Max-Product Belief Propagation

arXiv: https://arxiv.org/abs/2302.00099 · HTML: https://mlsanity.com/paper/2302.00099/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 27 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Loopy Belief Propagation for Approximate Inference: An Empirical Study](https://mlsanity.com/paper/1301.6725/) (2013)
- [Boolean Matrix Factorization and Noisy Completion via Message Passing](https://mlsanity.com/paper/1509.08535/) (2015)
- [Perturb-and-max-product: Sampling and learning in discrete energy-based models](https://mlsanity.com/paper/2111.02458/) (2021)
- [PGMax: Factor Graphs for Discrete Probabilistic Graphical Models and Loopy Belief Propagation in JAX](https://mlsanity.com/paper/2202.04110/) (2022)
- [Graphical Models with Attention for Context-Specific Independence and an Application to Perceptual Grouping](https://mlsanity.com/paper/2112.03371/) (2021)
- [Unsupervised Learning of Noisy-Or Bayesian Networks](https://mlsanity.com/paper/1309.6834/) (2013)

## Abstract (arXiv)

Noisy-OR Bayesian Networks (BNs) are a family of probabilistic graphical models which express rich statistical dependencies in binary data. Variational inference (VI) has been the main method proposed to learn noisy-OR BNs with complex latent structures (Jaakkola & Jordan, 1999; Ji et al., 2020; Buhai et al., 2020). However, the proposed VI approaches either (a) use a recognition network with standard amortized inference that cannot induce ``explaining-away''; or (b) assume a simple mean-field (MF) posterior which is vulnerable to bad local optima. Existing MF VI methods also update the MF parameters sequentially which makes them inherently slow. In this paper, we propose parallel max-product as an alternative algorithm for learning noisy-OR BNs with complex latent structures and we derive a fast stochastic training scheme that scales to large datasets. We evaluate both approaches on several benchmarks where VI is the state-of-the-art and show that our method (a) achieves better test performance than Ji et al. (2020) for learning noisy-OR BNs with hierarchical latent structures on large sparse real datasets; (b) recovers a higher number of ground truth parameters than Buhai et al. (2020) from cluttered synthetic scenes; and (c) solves the 2D blind deconvolution problem from Lazaro-Gredilla et al. (2021) and variant - including binary matrix factorization - while VI catastrophically fails and is up to two orders of magnitude slower.