# Gradient conjugate priors and multi-layer neural networks

arXiv: https://arxiv.org/abs/1802.02643 · HTML: https://mlsanity.com/paper/1802.02643/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Feb 2018.

## Built on by (most cited first, 6 of 6)

- [The Unreasonable Effectiveness of Deep Evidential Regression](https://mlsanity.com/paper/2205.10060/) (2022)
- [Single-model uncertainty quantification in neural network potentials does not consistently outperform model ensembles](https://mlsanity.com/paper/2305.01754/) (2023)
- [Multivariate Deep Evidential Regression](https://mlsanity.com/paper/2104.06135/) (2021)
- [Improving evidential deep learning via multi-task learning](https://mlsanity.com/paper/2112.09368/) (2021)
- [Deep Evidential Learning for Bayesian Quantile Regression](https://mlsanity.com/paper/2308.10650/) (2023)
- [Dr. Neurosymbolic, or: How I Learned to Stop Worrying and Accept Statistics](https://mlsanity.com/paper/2209.04049/) (2022)

## Abstract (arXiv)

The paper deals with learning probability distributions of observed data by artificial neural networks. We suggest a so-called gradient conjugate prior (GCP) update appropriate for neural networks, which is a modification of the classical Bayesian update for conjugate priors. We establish a connection between the gradient conjugate prior update and the maximization of the log-likelihood of the predictive distribution. Unlike for the Bayesian neural networks, we use deterministic weights of neural networks, but rather assume that the ground truth distribution is normal with unknown mean and variance and learn by the neural networks the parameters of a prior (normal-gamma distribution) for these unknown mean and variance. The update of the parameters is done, using the gradient that, at each step, directs towards minimizing the Kullback--Leibler divergence from the prior to the posterior distribution (both being normal-gamma). We obtain a corresponding dynamical system for the prior's parameters and analyze its properties. In particular, we study the limiting behavior of all the prior's parameters and show how it differs from the case of the classical full Bayesian update. The results are validated on synthetic and real world data sets.