# Bayesian Learning of Neural Network Architectures

arXiv: https://arxiv.org/abs/1901.04436 · HTML: https://mlsanity.com/paper/1901.04436/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2019.

## Built on by (most cited first, 5 of 5)

- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [Variational Inference for Infinitely Deep Neural Networks](https://mlsanity.com/paper/2209.10091/) (2022)
- [Latent Time Neural Ordinary Differential Equations](https://mlsanity.com/paper/2112.12728/) (2021)
- [DREAM: Debugging and Repairing AutoML Pipelines](https://mlsanity.com/paper/2401.00379/) (2023)
- [Deep Active Learning with Structured Neural Depth Search](https://mlsanity.com/paper/2306.02808/) (2023)

## Abstract (arXiv)

In this paper we propose a Bayesian method for estimating architectural parameters of neural networks, namely layer size and network depth. We do this by learning concrete distributions over these parameters. Our results show that regular networks with a learnt structure can generalise better on small datasets, while fully stochastic networks can be more robust to parameter initialisation. The proposed method relies on standard neural variational learning and, unlike randomised architecture search, does not require a retraining of the model, thus keeping the computational overhead at minimum.