# Parsimonious Bayesian deep networks

arXiv: https://arxiv.org/abs/1805.08719 · HTML: https://mlsanity.com/paper/1805.08719/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2018.

## Built on by (most cited first, 2 of 2)

- [A Variational Edge Partition Model for Supervised Graph Representation Learning](https://mlsanity.com/paper/2202.03233/) (2022)
- [Negative-Binomial Randomized Gamma Markov Processes for Heterogeneous Overdispersed Count Time Series](https://mlsanity.com/paper/2402.18995/) (2024)

## Abstract (arXiv)

Combining Bayesian nonparametrics and a forward model selection strategy, we construct parsimonious Bayesian deep networks (PBDNs) that infer capacity-regularized network architectures from the data and require neither cross-validation nor fine-tuning when training the model. One of the two essential components of a PBDN is the development of a special infinite-wide single-hidden-layer neural network, whose number of active hidden units can be inferred from the data. The other one is the construction of a greedy layer-wise learning algorithm that uses a forward model selection criterion to determine when to stop adding another hidden layer. We develop both Gibbs sampling and stochastic gradient descent based maximum a posteriori inference for PBDNs, providing state-of-the-art classification accuracy and interpretable data subtypes near the decision boundaries, while maintaining low computational complexity for out-of-sample prediction.