# Faithful Inversion of Generative Models for Effective Amortized Inference

arXiv: https://arxiv.org/abs/1712.00287 · HTML: https://mlsanity.com/paper/1712.00287/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2017.

## Built on by (most cited first, 29 of 29)

- [Tutorial on amortized optimization](https://mlsanity.com/paper/2202.00665/) (2022)
- [All-in-one simulation-based inference](https://mlsanity.com/paper/2404.09636/) (2024)
- [Learning nonparametric latent causal graphs with unknown interventions](https://mlsanity.com/paper/2306.02899/) (2023)
- [Amortized Variational Inference: When and Why?](https://mlsanity.com/paper/2307.11018/) (2023)
- [On Incorporating Inductive Biases into VAEs](https://mlsanity.com/paper/2106.13746/) (2021)
- [Graphically Structured Diffusion Models](https://mlsanity.com/paper/2210.11633/) (2022)
- [Tree Variational Autoencoders](https://mlsanity.com/paper/2306.08984/) (2023)
- [A Unified View on Solving Objective Mismatch in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2310.06253/) (2023)
- [ADAVI: Automatic Dual Amortized Variational Inference Applied To Pyramidal Bayesian Models](https://mlsanity.com/paper/2106.12248/) (2021)
- [Neuro-Causal Factor Analysis](https://mlsanity.com/paper/2305.19802/) (2023)
- [Provably Scalable Black-Box Variational Inference with Structured Variational Families](https://mlsanity.com/paper/2401.10989/) (2024)
- [Fast filtering of non-Gaussian models using Amortized Optimal Transport Maps](https://mlsanity.com/paper/2503.12633/) (2025)
- [Modeling Item Response Theory with Stochastic Variational Inference](https://mlsanity.com/paper/2108.11579/) (2021)
- [Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm](https://mlsanity.com/paper/2408.05834/) (2024)
- [Graphical Residual Flows](https://mlsanity.com/paper/2204.11846/) (2022)
- [PAVI: Plate-Amortized Variational Inference](https://mlsanity.com/paper/2206.05111/) (2022)
- [Computing with Categories in Machine Learning](https://mlsanity.com/paper/2303.04156/) (2023)
- [Rethinking Variational Inference for Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2311.00594/) (2023)
- [Inversion of Bayesian Networks](https://mlsanity.com/paper/2212.10649/) (2022)
- [On the Natural Gradient of the Evidence Lower Bound](https://mlsanity.com/paper/2307.11249/) (2023)
- [PAVI: Plate-Amortized Variational Inference](https://mlsanity.com/paper/2308.16022/) (2023)
- [Causal Posterior Estimation](https://mlsanity.com/paper/2505.21468/) (2025)
- [Intervening to Learn and Compose Causally Disentangled Representations](https://mlsanity.com/paper/2507.04754/) (2025)
- [Neural Posterior Estimation with Autoregressive Tiling for Detecting Objects in Astronomical Images](https://mlsanity.com/paper/2510.03074/) (2025)
- [Amortized Bayesian Inference on Multilevel Models of Arbitrary Structure](https://mlsanity.com/paper/2609.40024/) (2026)
- [SIReN-VAE: Leveraging Flows and Amortized Inference for Bayesian Networks](https://mlsanity.com/paper/2204.11847/) (2022)
- [A Probabilistic Generative Model of Free Categories](https://mlsanity.com/paper/2205.04545/) (2022)
- [DTD-VAE: Disentangled Temporal Dependencies VAE for Credit Risk Prediction](https://mlsanity.com/paper/2608.26473/) (2026)
- [Semi-Amortized Bayesian Inference with Reusable Local Posteriors](https://mlsanity.com/paper/2610.18945/) (2026)

## Abstract (arXiv)

Inference amortization methods share information across multiple posterior-inference problems, allowing each to be carried out more efficiently. Generally, they require the inversion of the dependency structure in the generative model, as the modeller must learn a mapping from observations to distributions approximating the posterior. Previous approaches have involved inverting the dependency structure in a heuristic way that fails to capture these dependencies correctly, thereby limiting the achievable accuracy of the resulting approximations. We introduce an algorithm for faithfully, and minimally, inverting the graphical model structure of any generative model. Such inverses have two crucial properties: (a) they do not encode any independence assertions that are absent from the model and; (b) they are local maxima for the number of true independencies encoded. We prove the correctness of our approach and empirically show that the resulting minimally faithful inverses lead to better inference amortization than existing heuristic approaches.