mlsanityData to 7 Oct 2026

Expressive Priors in Bayesian Neural Networks: Kernel Combinations and Periodic Functions

arXiv 1905.06076 ↗First posted 15 May 2019stat.ML cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

A simple, flexible approach to creating expressive priors in Gaussian process (GP) models makes new kernels from a combination of basic kernels, e.g. summing a periodic and linear kernel can capture seasonal variation with a long term trend. Despite a well-studied link between GPs and Bayesian neural networks (BNNs), the BNN analogue of this has not yet been explored. This paper derives BNN architectures mirroring such kernel combinations. Furthermore, it shows how BNNs can produce periodic kernels, which are often useful in this context. These ideas provide a principled approach to designing BNNs that incorporate prior knowledge about a function. We showcase the practical value of these ideas with illustrative experiments in supervised and reinforcement learning settings.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 6; 2023: 2; 2024: 7; 2025: 2.

22 papers.

Sort:

  1. 2021Bayesian Neural Network Priors Revisitedcited by 121
  2. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  3. 2024Variational Bayesian Last Layerscited by 67
  4. 2021Periodic Activation Functions Induce Stationaritycited by 21
  5. 2022Fast Neural Kernel Embeddings for General Activationscited by 18
  6. 2023Squared Neural Families: A New Class of Tractable Density Modelscited by 12
  7. 2024FSP-Laplace: Function-Space Priors for the Laplace Approximation in Bayesian Deep Learningcited by 11
  8. 2024Scalable Spatiotemporal Prediction with Bayesian Neural Fieldscited by 10
  9. 2021On out-of-distribution detection with Bayesian neural networkscited by 6
  10. 2022Simple initialization and parametrization of sinusoidal networks via their kernel bandwidthcited by 6
  11. 2024Alpha-VI DeepONet: A prior-robust variational Bayesian approach for enhancing DeepONets with uncertainty quantificationcited by 4
  12. 2023Bayesian Numerical Integration with Neural Networkscited by 3
10 more
  1. 2024Regularized KL-Divergence for Well-Defined Function-Space Variational Inference in Bayesian neural networkscited by 3
  2. 2022Incorporating Prior Knowledge into Neural Networks through an Implicit Composite Kernelcited by 2
  3. 2022A connection between probability, physics and neural networkscited by 2
  4. 2024Revisiting the Equivalence of Bayesian Neural Networks and Gaussian Processes: On the Importance of Learning Activationscited by 2
  5. 2021Conditional Deep Gaussian Processes: empirical Bayes hyperdata learningcited by 1
  6. 2022Augmenting Neural Networks with Priors on Function Valuescited by 1
  7. 2022On Connecting Deep Trigonometric Networks with Deep Gaussian Processes: Covariance, Expressivity, and Neural Tangent Kernelcited by 1
  8. 2024Thermodynamic limit in learning period threecited by 1
  9. 2025Bayesian neural networks with interpretable priors from Mercer kernelscited by 1
  10. 2025Hi-fi functional priors by learning activationscited by 0