# VI-DGP: A variational inference method with deep generative prior for solving high-dimensional inverse problems

arXiv: https://arxiv.org/abs/2302.11173 · HTML: https://mlsanity.com/paper/2302.11173/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 63 references that are in the map.
- First posted on arXiv 22 Feb 2023.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Advances in Variational Inference](https://mlsanity.com/paper/1711.05597/) (2017)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Modeling the Dynamics of PDE Systems with Physics-Constrained Deep Auto-Regressive Networks](https://mlsanity.com/paper/1906.05747/) (2019)
- [Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference](https://mlsanity.com/paper/1703.09194/) (2017)
- [Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification](https://mlsanity.com/paper/1802.00850/) (2018)
- [Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media](https://mlsanity.com/paper/1807.00882/) (2018)
- [Deep Generative Modeling for Mechanistic-based Learning and Design of Metamaterial Systems](https://mlsanity.com/paper/2006.15274/) (2020)
- [Solving inverse problems using conditional invertible neural networks](https://mlsanity.com/paper/2007.15849/) (2020)
- [SwitchNet: a neural network model for forward and inverse scattering problems](https://mlsanity.com/paper/1810.09675/) (2018)
- [Training-image based geostatistical inversion using a spatial generative adversarial neural network](https://mlsanity.com/paper/1708.04975/) (2017)
- [Solution of Physics-based Bayesian Inverse Problems with Deep Generative Priors](https://mlsanity.com/paper/2107.02926/) (2021)
- [Iterative Surrogate Model Optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks](https://mlsanity.com/paper/2008.05730/) (2020)
- [Deep autoregressive neural networks for high-dimensional inverse problems in groundwater contaminant source identification](https://mlsanity.com/paper/1812.09444/) (2018)
- [Fast PDE-constrained optimization via self-supervised operator learning](https://mlsanity.com/paper/2110.13297/) (2021)
- [Inversion using a new low-dimensional representation of complex binary geological media based on a deep neural network](https://mlsanity.com/paper/1710.09196/) (2017)
- [Data-Driven Model Reduction for the Bayesian Solution of Inverse Problems](https://mlsanity.com/paper/1403.4290/) (2014)
- [Bayesian multiscale deep generative model for the solution of high-dimensional inverse problems](https://mlsanity.com/paper/2102.03169/) (2021)
- [On Biased Stochastic Gradient Estimation](https://mlsanity.com/paper/1906.01133/) (2019)
- [Solving Inverse Wave Scattering with Deep Learning](https://mlsanity.com/paper/1911.13202/) (2019)
- [Approximate Bayesian Model Inversion for PDEs with Heterogeneous and State-Dependent Coefficients](https://mlsanity.com/paper/1902.06718/) (2019)
- [A domain-decomposed VAE method for Bayesian inverse problems](https://mlsanity.com/paper/2301.05708/) (2023)

## Built on by (most cited first, 4 of 4)

- [A Primer on Variational Inference for Physics-Informed Deep Generative Modelling](https://mlsanity.com/paper/2409.06560/) (2024)
- [Noise-Adaptive Diffusion Sampling for Inverse Problems Without Task-Specific Tuning](https://mlsanity.com/paper/2604.16919/) (2026)
- [Dimension-reduced KRnet maps for high-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2303.00573/) (2023)
- [Deep Adaptive Dimension Reduction for Bayesian Inference in Inverse Problems](https://mlsanity.com/paper/2605.29373/) (2026)

## Abstract (arXiv)

Solving high-dimensional Bayesian inverse problems (BIPs) with the variational inference (VI) method is promising but still challenging. The main difficulties arise from two aspects. First, VI methods approximate the posterior distribution using a simple and analytic variational distribution, which makes it difficult to estimate complex spatially-varying parameters in practice. Second, VI methods typically rely on gradient-based optimization, which can be computationally expensive or intractable when applied to BIPs involving partial differential equations (PDEs). To address these challenges, we propose a novel approximation method for estimating the high-dimensional posterior distribution. This approach leverages a deep generative model to learn a prior model capable of generating spatially-varying parameters. This enables posterior approximation over the latent variable instead of the complex parameters, thus improving estimation accuracy. Moreover, to accelerate gradient computation, we employ a differentiable physics-constrained surrogate model to replace the adjoint method. The proposed method can be fully implemented in an automatic differentiation manner. Numerical examples demonstrate two types of log-permeability estimation for flow in heterogeneous media. The results show the validity, accuracy, and high efficiency of the proposed method.