# LazyDINO: Fast, scalable, and efficiently amortized Bayesian inversion via structure-exploiting and surrogate-driven measure transport

arXiv: https://arxiv.org/abs/2411.12726 · HTML: https://mlsanity.com/paper/2411.12726/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 120 references that are in the map.
- First posted on arXiv 19 Nov 2024.

## Builds on (most cited first, 28 of 28)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Automatic Posterior Transformation for Likelihood-Free Inference](https://mlsanity.com/paper/1905.07488/) (2019)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Unconstrained Monotonic Neural Networks](https://mlsanity.com/paper/1908.05164/) (2019)
- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [Block Neural Autoregressive Flow](https://mlsanity.com/paper/1904.04676/) (2019)
- [LNO: Laplace Neural Operator for Solving Differential Equations](https://mlsanity.com/paper/2303.10528/) (2023)
- [Inference via low-dimensional couplings](https://mlsanity.com/paper/1703.06131/) (2017)
- [Nonlocality and Nonlinearity Implies Universality in Operator Learning](https://mlsanity.com/paper/2304.13221/) (2023)
- [On the representation and learning of monotone triangular transport maps](https://mlsanity.com/paper/2009.10303/) (2020)
- [Variational Autoencoding Neural Operators](https://mlsanity.com/paper/2302.10351/) (2023)
- [Efficient PDE-Constrained optimization under high-dimensional uncertainty using derivative-informed neural operators](https://mlsanity.com/paper/2305.20053/) (2023)
- [Derivative-enhanced Deep Operator Network](https://mlsanity.com/paper/2402.19242/) (2024)
- [Derivative-informed neural operator acceleration of geometric MCMC for infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2403.08220/) (2024)
- [Sequential Neural Methods for Likelihood-free Inference](https://mlsanity.com/paper/1811.08723/) (2018)
- [Bayesian model calibration for block copolymer self-assembly: Likelihood-free inference and expected information gain computation via measure transport](https://mlsanity.com/paper/2206.11343/) (2022)
- [Accurate, scalable, and efficient Bayesian optimal experimental design with derivative-informed neural operators](https://mlsanity.com/paper/2312.14810/) (2023)
- [Neural Posterior Estimation with Differentiable Simulators](https://mlsanity.com/paper/2207.05636/) (2022)
- [Sequential infinite-dimensional Bayesian optimal experimental design with derivative-informed latent attention neural operator](https://mlsanity.com/paper/2409.09141/) (2024)
- [Inference of Heterogeneous Material Properties via Infinite-Dimensional Integrated DIC](https://mlsanity.com/paper/2408.10217/) (2024)
- [Greedy inference with structure-exploiting lazy maps](https://mlsanity.com/paper/1906.00031/) (2019)
- [Fast Unconstrained Optimization via Hessian Averaging and Adaptive Gradient Sampling Methods](https://mlsanity.com/paper/2408.07268/) (2024)

## Built on by (most cited first, 7 of 7)

- [Performance of Neural and Polynomial Operator Surrogates](https://mlsanity.com/paper/2604.00689/) (2026)
- [Verification and Validation for Trustworthy Scientific Machine Learning](https://mlsanity.com/paper/2502.15496/) (2025)
- [A friendly introduction to triangular transport](https://mlsanity.com/paper/2503.21673/) (2025)
- [Deep Hilbert--Galerkin Methods for Infinite-Dimensional PDEs and Optimal Control](https://mlsanity.com/paper/2603.19463/) (2026)
- [Optimal Experimental Design for Reliable Learning of History-Dependent Constitutive Laws](https://mlsanity.com/paper/2603.12365/) (2026)
- [Universal Approximation of Nonlinear Operators and Their Derivatives](https://mlsanity.com/paper/2605.15285/) (2026)
- [Energy-based Transport for Amortized Bayesian Inference](https://mlsanity.com/paper/2605.15407/) (2026)

## Abstract (arXiv)

We present LazyDINO, a transport map variational inference method for fast, scalable, and efficiently amortized solutions of high-dimensional nonlinear Bayesian inverse problems with expensive parameter-to-observable (PtO) maps. Our method consists of an offline phase in which we construct a derivative-informed neural surrogate of the PtO map using joint samples of the PtO map and its Jacobian. During the online phase, when given observational data, we seek rapid posterior approximation using surrogate-driven training of a lazy map [Brennan et al., NeurIPS, (2020)], i.e., a structure-exploiting transport map with low-dimensional nonlinearity. The trained lazy map then produces approximate posterior samples or density evaluations. Our surrogate construction is optimized for amortized Bayesian inversion using lazy map variational inference. We show that (i) the derivative-based reduced basis architecture [O'Leary-Roseberry et al., Comput. Methods Appl. Mech. Eng., 388 (2022)] minimizes the upper bound on the expected error in surrogate posterior approximation, and (ii) the derivative-informed training formulation [O'Leary-Roseberry et al., J. Comput. Phys., 496 (2024)] minimizes the expected error due to surrogate-driven transport map optimization. Our numerical results demonstrate that LazyDINO is highly efficient in cost amortization for Bayesian inversion. We observe one to two orders of magnitude reduction of offline cost for accurate posterior approximation, compared to simulation-based amortized inference via conditional transport and conventional surrogate-driven transport. In particular, LazyDINO outperforms Laplace approximation consistently using fewer than 1000 offline samples, while other amortized inference methods struggle and sometimes fail at 16,000 offline samples.