# Gamma Belief Networks

arXiv: https://arxiv.org/abs/1512.03081 · HTML: https://mlsanity.com/paper/1512.03081/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2015.

## Built on by (most cited first, 19 of 19)

- [Topic Modelling Meets Deep Neural Networks: A Survey](https://mlsanity.com/paper/2103.00498/) (2021)
- [Representing Mixtures of Word Embeddings with Mixtures of Topic Embeddings](https://mlsanity.com/paper/2203.01570/) (2022)
- [Bayesian Attention Belief Networks](https://mlsanity.com/paper/2106.05251/) (2021)
- [Learning Prototype-oriented Set Representations for Meta-Learning](https://mlsanity.com/paper/2110.09140/) (2021)
- [HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding](https://mlsanity.com/paper/2210.10625/) (2022)
- [LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Models](https://mlsanity.com/paper/2406.09008/) (2024)
- [Neural Topic Modeling with Large Language Models in the Loop](https://mlsanity.com/paper/2411.08534/) (2024)
- [TopicNet: Semantic Graph-Guided Topic Discovery](https://mlsanity.com/paper/2110.14286/) (2021)
- [Fixed points of nonnegative neural networks](https://mlsanity.com/paper/2106.16239/) (2021)
- [A Non-negative VAE:the Generalized Gamma Belief Network](https://mlsanity.com/paper/2408.03388/) (2024)
- [Broad Spectrum Structure Discovery in Large-Scale Higher-Order Networks](https://mlsanity.com/paper/2505.21748/) (2025)
- [Matching Visual Features to Hierarchical Semantic Topics for Image Paragraph Captioning](https://mlsanity.com/paper/2105.04143/) (2021)
- [Beyond Spectral Decomposition: Bayesian Contrastive Learning and its Non-negative Formulation via Factor Analysis](https://mlsanity.com/paper/2407.21740/) (2024)
- [Negative-Binomial Randomized Gamma Markov Processes for Heterogeneous Overdispersed Count Time Series](https://mlsanity.com/paper/2402.18995/) (2024)
- [Scalable Weibull Graph Attention Autoencoder for Modeling Document Networks](https://mlsanity.com/paper/2410.09696/) (2024)
- [Multinomial belief networks for healthcare data](https://mlsanity.com/paper/2311.16909/) (2023)
- [Scaling up Dynamic Edge Partition Models via Stochastic Gradient MCMC](https://mlsanity.com/paper/2403.00044/) (2024)
- [Hierarchical Graph Topic Modeling with Topic Tree-based Transformer](https://mlsanity.com/paper/2502.11345/) (2025)
- [Bayesian Poisson-Randomized Gamma Tensor Factorization with Application to International Trade Flows](https://mlsanity.com/paper/2606.17267/) (2026)

## Abstract (arXiv)

To infer multilayer deep representations of high-dimensional discrete and nonnegative real vectors, we propose an augmentable gamma belief network (GBN) that factorizes each of its hidden layers into the product of a sparse connection weight matrix and the nonnegative real hidden units of the next layer. The GBN's hidden layers are jointly trained with an upward-downward Gibbs sampler that solves each layer with the same subroutine. The gamma-negative binomial process combined with a layer-wise training strategy allows inferring the width of each layer given a fixed budget on the width of the first layer. Example results illustrate interesting relationships between the width of the first layer and the inferred network structure, and demonstrate that the GBN can add more layers to improve its performance in both unsupervisedly extracting features and predicting heldout data. For exploratory data analysis, we extract trees and subnetworks from the learned deep network to visualize how the very specific factors discovered at the first hidden layer and the increasingly more general factors discovered at deeper hidden layers are related to each other, and we generate synthetic data by propagating random variables through the deep network from the top hidden layer back to the bottom data layer.