# Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic Functions

arXiv: https://arxiv.org/abs/1905.06076 · HTML: https://mlsanity.com/paper/1905.06076/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2019.

## Built on by (most cited first, 22 of 22)

- [Bayesian Neural Network Priors Revisited](https://mlsanity.com/paper/2102.06571/) (2021)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Variational Bayesian Last Layers](https://mlsanity.com/paper/2404.11599/) (2024)
- [Periodic Activation Functions Induce Stationarity](https://mlsanity.com/paper/2110.13572/) (2021)
- [Fast Neural Kernel Embeddings for General Activations](https://mlsanity.com/paper/2209.04121/) (2022)
- [Squared Neural Families: A New Class of Tractable Density Models](https://mlsanity.com/paper/2305.13552/) (2023)
- [FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learning](https://mlsanity.com/paper/2407.13711/) (2024)
- [Scalable Spatiotemporal Prediction with Bayesian Neural Fields](https://mlsanity.com/paper/2403.07657/) (2024)
- [On out-of-distribution detection with Bayesian neural networks](https://mlsanity.com/paper/2110.06020/) (2021)
- [Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth](https://mlsanity.com/paper/2211.14503/) (2022)
- [Alpha-VI DeepONet: A prior-robust variational Bayesian approach for enhancing DeepONets with uncertainty quantification](https://mlsanity.com/paper/2408.00681/) (2024)
- [Bayesian Numerical Integration with Neural Networks](https://mlsanity.com/paper/2305.13248/) (2023)
- [Regularized KL-Divergence for Well-Defined Function-Space Variational Inference in Bayesian neural networks](https://mlsanity.com/paper/2406.04317/) (2024)
- [Incorporating Prior Knowledge into Neural Networks through an Implicit Composite Kernel](https://mlsanity.com/paper/2205.07384/) (2022)
- [A connection between probability, physics and neural networks](https://mlsanity.com/paper/2209.12737/) (2022)
- [Revisiting the Equivalence of Bayesian Neural Networks and Gaussian Processes: On the Importance of Learning Activations](https://mlsanity.com/paper/2410.15777/) (2024)
- [Conditional Deep Gaussian Processes: empirical Bayes hyperdata learning](https://mlsanity.com/paper/2110.00568/) (2021)
- [Augmenting Neural Networks with Priors on Function Values](https://mlsanity.com/paper/2202.04798/) (2022)
- [On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernel](https://mlsanity.com/paper/2203.07411/) (2022)
- [Thermodynamic limit in learning period three](https://mlsanity.com/paper/2405.08825/) (2024)
- [Bayesian neural networks with interpretable priors from Mercer kernels](https://mlsanity.com/paper/2510.23745/) (2025)
- [Hi-fi functional priors by learning activations](https://mlsanity.com/paper/2508.08880/) (2025)

## Abstract (arXiv)

A simple, flexible approach to creating expressive priors in Gaussian process (GP) models makes new kernels from a combination of basic kernels, e.g. summing a periodic and linear kernel can capture seasonal variation with a long term trend. Despite a well-studied link between GPs and Bayesian neural networks (BNNs), the BNN analogue of this has not yet been explored. This paper derives BNN architectures mirroring such kernel combinations. Furthermore, it shows how BNNs can produce periodic kernels, which are often useful in this context. These ideas provide a principled approach to designing BNNs that incorporate prior knowledge about a function. We showcase the practical value of these ideas with illustrative experiments in supervised and reinforcement learning settings.