# Bayesian deep learning with hierarchical prior: Predictions from limited and noisy data

arXiv: https://arxiv.org/abs/1907.04240 · HTML: https://mlsanity.com/paper/1907.04240/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jul 2019.

## Built on by (most cited first, 8 of 8)

- [Bayesian Physics-Informed Extreme Learning Machine for Forward and Inverse PDE Problems with Noisy Data](https://mlsanity.com/paper/2205.06948/) (2022)
- [Bayesian neural networks for weak solution of PDEs with uncertainty quantification](https://mlsanity.com/paper/2101.04879/) (2021)
- [A Bayesian Deep Learning Approach to Near-Term Climate Prediction](https://mlsanity.com/paper/2202.11244/) (2022)
- [GEM: Group Enhanced Model for Learning Dynamical Control Systems](https://mlsanity.com/paper/2104.02844/) (2021)
- [Bayesian sparsification for deep neural networks with Bayesian model reduction](https://mlsanity.com/paper/2309.12095/) (2023)
- [Bayesian Deep Learning Hyperparameter Search for Robust Function Mapping to Polynomials with Noise](https://mlsanity.com/paper/2106.12532/) (2021)
- [A Machine Learning-based Characterization Framework for Parametric Representation of Nonlinear Sloshing](https://mlsanity.com/paper/2201.11663/) (2022)
- [Navigating Uncertainties in Machine Learning for Structural Dynamics: A Comprehensive Survey of Probabilistic and Non-Probabilistic Approaches in Forward and Inverse Problems](https://mlsanity.com/paper/2408.08629/) (2024)

## Abstract (arXiv)

Datasets in engineering applications are often limited and contaminated, mainly due to unavoidable measurement noise and signal distortion. Thus, using conventional data-driven approaches to build a reliable discriminative model, and further applying this identified surrogate to uncertainty analysis remains to be very challenging. A deep learning approach is presented to provide predictions based on limited and noisy data. To address noise perturbation, the Bayesian learning method that naturally facilitates an automatic updating mechanism is considered to quantify and propagate model uncertainties into predictive quantities. Specifically, hierarchical Bayesian modeling (HBM) is first adopted to describe model uncertainties, which allows the prior assumption to be less subjective, while also makes the proposed surrogate more robust. Next, the Bayesian inference is seamlessly integrated into the DL framework, which in turn supports probabilistic programming by yielding a probability distribution of the quantities of interest rather than their point estimates. Variational inference (VI) is implemented for the posterior distribution analysis where the intractable marginalization of the likelihood function over parameter space is framed in an optimization format, and stochastic gradient descent method is applied to solve this optimization problem. Finally, Monte Carlo simulation is used to obtain an unbiased estimator in the predictive phase of Bayesian inference, where the proposed Bayesian deep learning (BDL) scheme is able to offer confidence bounds for the output estimation by analyzing propagated uncertainties. The effectiveness of Bayesian shrinkage is demonstrated in improving predictive performance using contaminated data, and various examples are provided to illustrate concepts, methodologies, and algorithms of this proposed BDL modeling technique.