# Learning to solve Bayesian inverse problems: An amortized variational inference approach using Gaussian and Flow guides

arXiv: https://arxiv.org/abs/2305.20004 · HTML: https://mlsanity.com/paper/2305.20004/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 83 references that are in the map.
- First posted on arXiv 31 May 2023.

## Builds on (most cited first, 23 of 23)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Analyzing Inverse Problems with Invertible Neural Networks](https://mlsanity.com/paper/1808.04730/) (2018)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Automatic Differentiation Variational Inference](https://mlsanity.com/paper/1603.00788/) (2016)
- [Guided Image Generation with Conditional Invertible Neural Networks](https://mlsanity.com/paper/1907.02392/) (2019)
- [Inference Suboptimality in Variational Autoencoders](https://mlsanity.com/paper/1801.03558/) (2018)
- [Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification](https://mlsanity.com/paper/1802.00850/) (2018)
- [Challenges in Markov chain Monte Carlo for Bayesian neural networks](https://mlsanity.com/paper/1910.06539/) (2019)
- [Solving inverse problems using conditional invertible neural networks](https://mlsanity.com/paper/2007.15849/) (2020)
- [HINT: Hierarchical Invertible Neural Transport for Density Estimation and Bayesian Inference](https://mlsanity.com/paper/1905.10687/) (2019)
- [Personalized Radiotherapy Design for Glioblastoma: Integrating Mathematical Tumor Models, Multimodal Scans and Bayesian Inference](https://mlsanity.com/paper/1807.00499/) (2018)
- [Semi-supervised Invertible Neural Operators for Bayesian Inverse Problems](https://mlsanity.com/paper/2209.02772/) (2022)
- [Meta-Amortized Variational Inference and Learning](https://mlsanity.com/paper/1902.01950/) (2019)
- [Amortized Variational Inference: A Systematic Review](https://mlsanity.com/paper/2209.10888/) (2022)
- [Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction](https://mlsanity.com/paper/1510.06053/) (2015)
- [Scalable conditional deep inverse Rosenblatt transports using tensor-trains and gradient-based dimension reduction](https://mlsanity.com/paper/2106.04170/) (2021)
- [Bayesian Neural Networks With Maximum Mean Discrepancy Regularization](https://mlsanity.com/paper/2003.00952/) (2020)
- [Sparse Variational Bayesian Approximations for Nonlinear Inverse Problems: applications in nonlinear elastography](https://mlsanity.com/paper/1412.0473/) (2014)
- [Variational Reformulation of Bayesian Inverse Problems](https://mlsanity.com/paper/1410.5522/) (2014)

## Built on by (most cited first, 5 of 5)

- [Physics-Informed Latent Neural Operator for Real-time Predictions of time-dependent parametric PDEs](https://mlsanity.com/paper/2501.08428/) (2025)
- [Conditional Pseudo-Reversible Normalizing Flow for Surrogate Modeling in Quantifying Uncertainty Propagation](https://mlsanity.com/paper/2404.00502/) (2024)
- [Solving High-dimensional Inverse Problems Using Amortized Likelihood-free Inference with Noisy and Incomplete Data](https://mlsanity.com/paper/2412.04565/) (2024)
- [Energy-based Transport for Amortized Bayesian Inference](https://mlsanity.com/paper/2605.15407/) (2026)
- [MoRF-AST: Calibrated Probabilistic Virtual Sensing for Structural Monitoring under Changing Operating Conditions](https://mlsanity.com/paper/2608.24531/) (2026)

## Abstract (arXiv)

Inverse problems, i.e., estimating parameters of physical models from experimental data, are ubiquitous in science and engineering. The Bayesian formulation is the gold standard because it alleviates ill-posedness issues and quantifies epistemic uncertainty. Since analytical posteriors are not typically available, one resorts to Markov chain Monte Carlo sampling or approximate variational inference. However, inference needs to be rerun from scratch for each new set of data. This drawback limits the applicability of the Bayesian formulation to real-time settings, e.g., health monitoring of engineered systems, and medical diagnosis. The objective of this paper is to develop a methodology that enables real-time inference by learning the Bayesian inverse map, i.e., the map from data to posteriors. Our approach is as follows. We parameterize the posterior distribution as a function of data. This work outlines two distinct approaches to do this. The first method involves parameterizing the posterior using an amortized full-rank Gaussian guide, implemented through neural networks. The second method utilizes a Conditional Normalizing Flow guide, employing conditional invertible neural networks for cases where the target posterior is arbitrarily complex. In both approaches, we learn the network parameters by amortized variational inference which involves maximizing the expectation of evidence lower bound over all possible datasets compatible with the model. We demonstrate our approach by solving a set of benchmark problems from science and engineering. Our results show that the posterior estimates of our approach are in agreement with the corresponding ground truth obtained by Markov chain Monte Carlo. Once trained, our approach provides the posterior distribution for a given observation just at the cost of a forward pass of the neural network.