# Forward Amortized Inference for Likelihood-Free Variational Marginalization

arXiv: https://arxiv.org/abs/1805.11542 · HTML: https://mlsanity.com/paper/1805.11542/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2018.

## Built on by (most cited first, 14 of 14)

- [Amortized Inference for Causal Structure Learning](https://mlsanity.com/paper/2205.12934/) (2022)
- [A Diffusion Model Framework for Unsupervised Neural Combinatorial Optimization](https://mlsanity.com/paper/2406.01661/) (2024)
- [Conformal Contextual Robust Optimization](https://mlsanity.com/paper/2310.10003/) (2023)
- [Sequential Monte Carlo for Inclusive KL Minimization in Amortized Variational Inference](https://mlsanity.com/paper/2403.10610/) (2024)
- [Variational Inference for Deblending Crowded Starfields](https://mlsanity.com/paper/2102.02409/) (2021)
- [Variational Inference with Coverage Guarantees in Simulation-Based Inference](https://mlsanity.com/paper/2305.14275/) (2023)
- [FNOPE: Simulation-based inference on function spaces with Fourier Neural Operators](https://mlsanity.com/paper/2505.22573/) (2025)
- [Scalable Bayesian Inference for Detection and Deblending in Astronomical Images](https://mlsanity.com/paper/2207.05642/) (2022)
- [Globally Convergent Variational Inference](https://mlsanity.com/paper/2501.08201/) (2025)
- [Neural Posterior Estimation with Autoregressive Tiling for Detecting Objects in Astronomical Images](https://mlsanity.com/paper/2510.03074/) (2025)
- [Consistent Zero-Shot Imitation with Contrastive Goal Inference](https://mlsanity.com/paper/2510.17059/) (2025)
- [Statistical Inference for Coadded Astronomical Images](https://mlsanity.com/paper/2211.09300/) (2022)
- [Robust and highly scalable estimation of directional couplings from time-shifted signals](https://mlsanity.com/paper/2406.02545/) (2024)
- [Scalable Amortized Variational Inference for Non-Poisson Buy-'Til-You-Die Models](https://mlsanity.com/paper/2608.19022/) (2026)

## Abstract (arXiv)

In this paper, we introduce a new form of amortized variational inference by using the forward KL divergence in a joint-contrastive variational loss. The resulting forward amortized variational inference is a likelihood-free method as its gradient can be sampled without bias and without requiring any evaluation of either the model joint distribution or its derivatives. We prove that our new variational loss is optimized by the exact posterior marginals in the fully factorized mean-field approximation, a property that is not shared with the more conventional reverse KL inference. Furthermore, we show that forward amortized inference can be easily marginalized over large families of latent variables in order to obtain a marginalized variational posterior. We consider two examples of variational marginalization. In our first example we train a Bayesian forecaster for predicting a simplified chaotic model of atmospheric convection. In the second example we train an amortized variational approximation of a Bayesian optimal classifier by marginalizing over the model space. The result is a powerful meta-classification network that can solve arbitrary classification problems without further training.