# Scaling Up Bayesian Uncertainty Quantification for Inverse Problems using Deep Neural Networks

arXiv: https://arxiv.org/abs/2101.03906 · HTML: https://mlsanity.com/paper/2101.03906/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 111 references that are in the map.
- First posted on arXiv 11 Jan 2021.

## Builds on (most cited first, 21 of 21)

- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [A Review of Convolutional Neural Networks for Inverse Problems in Imaging](https://mlsanity.com/paper/1710.04011/) (2017)
- [MeshCNN: A Network with an Edge](https://mlsanity.com/paper/1809.05910/) (2018)
- [Accelerated Information Gradient flow](https://mlsanity.com/paper/1909.02102/) (2019)
- [Dimension-independent likelihood-informed MCMC](https://mlsanity.com/paper/1411.3688/) (2014)
- [Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks](https://mlsanity.com/paper/1808.03620/) (2018)
- [Data Assimilation: A Mathematical Introduction](https://mlsanity.com/paper/1506.07825/) (2015)
- [Information Newton's flow: second-order optimization method in probability space](https://mlsanity.com/paper/2001.04341/) (2020)
- [Deep Integro-Difference Equation Models for Spatio-Temporal Forecasting](https://mlsanity.com/paper/1910.13524/) (2019)
- [A regularizing iterative ensemble Kalman method for PDE-constrained inverse problems](https://mlsanity.com/paper/1505.03876/) (2015)
- [Tikhonov Regularization Within Ensemble Kalman Inversion](https://mlsanity.com/paper/1901.10382/) (2019)
- [Ensemble Kalman methods for high-dimensional hierarchical dynamic space-time models](https://mlsanity.com/paper/1704.06988/) (2017)
- [A Stable Manifold MCMC Method for High Dimensions](https://mlsanity.com/paper/1403.7711/) (2014)
- [Multi-resolution filters for massive spatio-temporal data](https://mlsanity.com/paper/1810.04200/) (2018)

## Built on by (most cited first, 10 of 10)

- [Uncertainty Quantification in Large Language Models Through Convex Hull Analysis](https://mlsanity.com/paper/2406.19712/) (2024)
- [Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization](https://mlsanity.com/paper/2205.13098/) (2022)
- [Solving multiphysics-based inverse problems with learned surrogates and constraints](https://mlsanity.com/paper/2307.11099/) (2023)
- [Uncertainty-aware Predict-Then-Optimize Framework for Equitable Post-Disaster Power Restoration](https://mlsanity.com/paper/2508.04780/) (2025)
- [Good Things Come in Pairs: Paired Autoencoders for Inverse Problems](https://mlsanity.com/paper/2505.06549/) (2025)
- [Cycle Consistency-based Uncertainty Quantification of Neural Networks in Inverse Imaging Problems](https://mlsanity.com/paper/2305.12852/) (2023)
- [Bayesian Learning via Q-Exponential Process](https://mlsanity.com/paper/2210.07987/) (2022)
- [Neural-Inspired Posterior Approximation (NIPA)](https://mlsanity.com/paper/2601.22539/) (2026)
- [High-Dimensional Simulation-Based Inference in Latent Spaces](https://mlsanity.com/paper/2609.37381/) (2026)
- [SCOPE Before You Sample: Conformal Scores Guide Posterior Exploration for Fast and Calibrated Bayesian Predictive Inference](https://mlsanity.com/paper/2610.17146/) (2026)

## Abstract (arXiv)

Due to the importance of uncertainty quantification (UQ), Bayesian approach to inverse problems has recently gained popularity in applied mathematics, physics, and engineering. However, traditional Bayesian inference methods based on Markov Chain Monte Carlo (MCMC) tend to be computationally intensive and inefficient for such high dimensional problems. To address this issue, several methods based on surrogate models have been proposed to speed up the inference process. More specifically, the calibration-emulation-sampling (CES) scheme has been proven to be successful in large dimensional UQ problems. In this work, we propose a novel CES approach for Bayesian inference based on deep neural network models for the emulation phase. The resulting algorithm is computationally more efficient and more robust against variations in the training set. Further, by using an autoencoder (AE) for dimension reduction, we have been able to speed up our Bayesian inference method up to three orders of magnitude. Overall, our method, henceforth called Dimension-Reduced Emulative Autoencoder Monte Carlo (DREAMC) algorithm, is able to scale Bayesian UQ up to thousands of dimensions for inverse problems. Using two low-dimensional (linear and nonlinear) inverse problems we illustrate the validity of this approach. Next, we apply our method to two high-dimensional numerical examples (elliptic and advection-diffussion) to demonstrate its computational advantages over existing algorithms.