# Refining Amortized Posterior Approximations using Gradient-Based Summary Statistics

arXiv: https://arxiv.org/abs/2305.08733 · HTML: https://mlsanity.com/paper/2305.08733/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 15 May 2023.

## Builds on (most cited first, 13 of 13)

- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [BayesFlow: Learning complex stochastic models with invertible neural networks](https://mlsanity.com/paper/2003.06281/) (2020)
- [Iterative Amortized Inference](https://mlsanity.com/paper/1807.09356/) (2018)
- [Deep Bayesian Inversion](https://mlsanity.com/paper/1811.05910/) (2018)
- [Reliable amortized variational inference with physics-based latent distribution correction](https://mlsanity.com/paper/2207.11640/) (2022)
- [Generalized Normalizing Flows via Markov Chains](https://mlsanity.com/paper/2111.12506/) (2021)
- [Preconditioned training of normalizing flows for variational inference in inverse problems](https://mlsanity.com/paper/2101.03709/) (2021)
- [Invert to Learn to Invert](https://mlsanity.com/paper/1911.10914/) (2019)
- [Amortized Normalizing Flows for Transcranial Ultrasound with Uncertainty Quantification](https://mlsanity.com/paper/2303.03478/) (2023)
- [Loop Unrolled Shallow Equilibrium Regularizer (LUSER) -- A Memory-Efficient Inverse Problem Solver](https://mlsanity.com/paper/2210.04987/) (2022)
- [Recurrent Localization Networks applied to the Lippmann-Schwinger Equation](https://mlsanity.com/paper/2102.00063/) (2021)

## Built on by (most cited first, 3 of 3)

- [WISE: full-Waveform variational Inference via Subsurface Extensions](https://mlsanity.com/paper/2401.06230/) (2023)
- [Solving multiphysics-based inverse problems with learned surrogates and constraints](https://mlsanity.com/paper/2307.11099/) (2023)
- [InvertibleNetworks.jl: A Julia package for scalable normalizing flows](https://mlsanity.com/paper/2312.13480/) (2023)

## Abstract (arXiv)

We present an iterative framework to improve the amortized approximations of posterior distributions in the context of Bayesian inverse problems, which is inspired by loop-unrolled gradient descent methods and is theoretically grounded in maximally informative summary statistics. Amortized variational inference is restricted by the expressive power of the chosen variational distribution and the availability of training data in the form of joint data and parameter samples, which often lead to approximation errors such as the amortization gap. To address this issue, we propose an iterative framework that refines the current amortized posterior approximation at each step. Our approach involves alternating between two steps: (1) constructing a training dataset consisting of pairs of summarized data residuals and parameters, where the summarized data residual is generated using a gradient-based summary statistic, and (2) training a conditional generative model -- a normalizing flow in our examples -- on this dataset to obtain a probabilistic update of the unknown parameter. This procedure leads to iterative refinement of the amortized posterior approximations without the need for extra training data. We validate our method in a controlled setting by applying it to a stylized problem, and observe improved posterior approximations with each iteration. Additionally, we showcase the capability of our method in tackling realistically sized problems by applying it to transcranial ultrasound, a high-dimensional, nonlinear inverse problem governed by wave physics, and observe enhanced posterior quality through better image reconstruction with the posterior mean.