# Empowering Bayesian Neural Networks with Functional Priors through Anchored Ensembling for Mechanics Surrogate Modeling Applications

arXiv: https://arxiv.org/abs/2409.05234 · HTML: https://mlsanity.com/paper/2409.05234/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 8 Sep 2024.

## Builds on (most cited first, 24 of 24)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Weight Uncertainty in Neural Networks](https://mlsanity.com/paper/1505.05424/) (2015)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [Randomized Prior Functions for Deep Reinforcement Learning](https://mlsanity.com/paper/1806.03335/) (2018)
- [Functional Variational Bayesian Neural Networks](https://mlsanity.com/paper/1903.05779/) (2019)
- [Bayesian Neural Network Priors Revisited](https://mlsanity.com/paper/2102.06571/) (2021)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems](https://mlsanity.com/paper/2205.08304/) (2022)
- [Understanding predictive information criteria for Bayesian models](https://mlsanity.com/paper/1307.5928/) (2013)
- [Scalable Uncertainty Quantification for Deep Operator Networks using Randomized Priors](https://mlsanity.com/paper/2203.03048/) (2022)
- [Learning Functional Priors and Posteriors from Data and Physics](https://mlsanity.com/paper/2106.05863/) (2021)
- [Solution of Physics-based Bayesian Inverse Problems with Deep Generative Priors](https://mlsanity.com/paper/2107.02926/) (2021)
- [Efficient Uncertainty Quantification for Dynamic Subsurface Flow with Surrogate by Theory-guided Neural Network](https://mlsanity.com/paper/2004.13560/) (2020)
- [Neural Network Training Using ℓ1-Regularization and Bi-fidelity Data](https://mlsanity.com/paper/2105.13011/) (2021)
- [On sparse regression, Lp-regularization, and automated model discovery](https://mlsanity.com/paper/2310.06872/) (2023)
- [Neural network-based multiscale modeling of finite strain magneto-elasticity with relaxed convexity criteria](https://mlsanity.com/paper/2308.16311/) (2023)
- [A dimension-reduced variational approach for solving physics-based inverse problems using generative adversarial network priors and normalizing flows](https://mlsanity.com/paper/2310.04690/) (2023)
- [Bayesian Neural Networks with Domain Knowledge Priors](https://mlsanity.com/paper/2402.13410/) (2024)
- [Tractable Approximate Gaussian Inference for Bayesian Neural Networks](https://mlsanity.com/paper/2004.09281/) (2020)
- [Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles](https://mlsanity.com/paper/2302.09748/) (2023)
- [A Framework for Strategic Discovery of Credible Neural Network Surrogate Models under Uncertainty](https://mlsanity.com/paper/2403.08901/) (2024)
- [Surrogate Neural Network Model for Sensitivity Analysis and Uncertainty Quantification of the Mechanical Behavior in the Optical Lens-Barrel Assembly](https://mlsanity.com/paper/2201.09659/) (2022)

## Built on by (most cited first, 2 of 2)

- [Quantifying Out-of-Training Uncertainty of Neural-Network based Turbulence Closures](https://mlsanity.com/paper/2508.16891/) (2025)
- [Functional-prior-based approaches to Bayesian PDE-constrained inversion using physics-informed neural networks](https://mlsanity.com/paper/2605.07060/) (2026)

## Abstract (arXiv)

In recent years, neural networks (NNs) have become increasingly popular for surrogate modeling tasks in mechanics and materials modeling applications. While traditional NNs are deterministic functions that rely solely on data to learn the input--output mapping, casting NN training within a Bayesian framework allows to quantify uncertainties, in particular epistemic uncertainties that arise from lack of training data, and to integrate a priori knowledge via the Bayesian prior. However, the high dimensionality and non-physicality of the NN parameter space, and the complex relationship between parameters (NN weights) and predicted outputs, renders both prior design and posterior inference challenging. In this work we present a novel BNN training scheme based on anchored ensembling that can integrate a priori information available in the function space, from e.g. low-fidelity models. The anchoring scheme makes use of low-rank correlations between NN parameters, learnt from pre-training to realizations of the functional prior. We also perform a study to demonstrate how correlations between NN weights, which are often neglected in existing BNN implementations, is critical to appropriately transfer knowledge between the function-space and parameter-space priors. Performance of our novel BNN algorithm is first studied on a small 1D example to illustrate the algorithm's behavior in both interpolation and extrapolation settings. Then, a thorough assessment is performed on a multi--input--output materials surrogate modeling example, where we demonstrate the algorithm's capabilities both in terms of accuracy and quality of the uncertainty estimation, for both in-distribution and out-of-distribution data.