# Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference

arXiv: https://arxiv.org/abs/2310.04395 · HTML: https://mlsanity.com/paper/2310.04395/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 6 Oct 2023.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 30 of 32)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows](https://mlsanity.com/paper/1805.07226/) (2018)
- [Automatic Posterior Transformation for Likelihood-Free Inference](https://mlsanity.com/paper/1905.07488/) (2019)
- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [BayesFlow: Learning complex stochastic models with invertible neural networks](https://mlsanity.com/paper/2003.06281/) (2020)
- [Validating Bayesian Inference Algorithms with Simulation-Based Calibration](https://mlsanity.com/paper/1804.06788/) (2018)
- [Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design](https://mlsanity.com/paper/2103.02438/) (2021)
- [Real-time gravitational-wave science with neural posterior estimation](https://mlsanity.com/paper/2106.12594/) (2021)
- [Simulation Intelligence: Towards a New Generation of Scientific Methods](https://mlsanity.com/paper/2112.03235/) (2021)
- [Mining gold from implicit models to improve likelihood-free inference](https://mlsanity.com/paper/1805.12244/) (2018)
- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models](https://mlsanity.com/paper/2210.04872/) (2022)
- [Compositional Score Modeling for Simulation-based Inference](https://mlsanity.com/paper/2209.14249/) (2022)
- [Variational methods for simulation-based inference](https://mlsanity.com/paper/2203.04176/) (2022)
- [JANA: Jointly Amortized Neural Approximation of Complex Bayesian Models](https://mlsanity.com/paper/2302.09125/) (2023)
- [MadMiner: Machine learning-based inference for particle physics](https://mlsanity.com/paper/1907.10621/) (2019)
- [Sequential Neural Posterior and Likelihood Approximation](https://mlsanity.com/paper/2102.06522/) (2021)
- [Neural Importance Sampling for Rapid and Reliable Gravitational-Wave Inference](https://mlsanity.com/paper/2210.05686/) (2022)
- [Graphical Test for Discrete Uniformity and its Applications in Goodness of Fit Evaluation and Multiple Sample Comparison](https://mlsanity.com/paper/2103.10522/) (2021)
- [A Guide to Constraining Effective Field Theories with Machine Learning](https://mlsanity.com/paper/1805.00020/) (2018)
- [Score Matched Neural Exponential Families for Likelihood-Free Inference](https://mlsanity.com/paper/2012.10903/) (2020)
- [Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy](https://mlsanity.com/paper/2209.02439/) (2022)
- [A Review of Change of Variable Formulas for Generative Modeling](https://mlsanity.com/paper/2308.02652/) (2023)

## Built on by (most cited first, 14 of 14)

- [Consistency Models for Scalable and Fast Simulation-Based Inference](https://mlsanity.com/paper/2312.05440/) (2023)
- [Amortized Bayesian Multilevel Models](https://mlsanity.com/paper/2408.13230/) (2024)
- [Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data](https://mlsanity.com/paper/2501.13483/) (2025)
- [Multifidelity Simulation-based Inference for Computationally Expensive Simulators](https://mlsanity.com/paper/2502.08416/) (2025)
- [BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Python](https://mlsanity.com/paper/2602.07098/) (2026)
- [Low-Budget Simulation-Based Inference with Bayesian Neural Networks](https://mlsanity.com/paper/2408.15136/) (2024)
- [Multilevel neural simulation-based inference](https://mlsanity.com/paper/2506.06087/) (2025)
- [Improving the Accuracy of Amortized Model Comparison with Self-Consistency](https://mlsanity.com/paper/2508.20614/) (2025)
- [Amortized Bayesian Workflow](https://mlsanity.com/paper/2409.04332/) (2024)
- [An Introduction to Bayesian and Frequentist Simulation-Based Inference with Machine Learning](https://mlsanity.com/paper/2607.21702/) (2026)
- [Improving the Accuracy of Amortized Model Comparison with Self-Consistency](https://mlsanity.com/paper/2512.14308/) (2025)
- [Preconditioned Robust Neural Posterior Estimation for Misspecified Simulators](https://mlsanity.com/paper/2602.18004/) (2026)
- [Unsupervised Continual Learning for Amortized Bayesian Inference](https://mlsanity.com/paper/2602.22884/) (2026)
- [Information-Preserving Domain Transfer with Unlabeled Data in Misspecified Simulation-Based Inference](https://mlsanity.com/paper/2605.05652/) (2026)

## Abstract (arXiv)

We propose a method to improve the efficiency and accuracy of amortized Bayesian inference by leveraging universal symmetries in the joint probabilistic model of parameters and data. In a nutshell, we invert Bayes' theorem and estimate the marginal likelihood based on approximate representations of the joint model. Upon perfect approximation, the marginal likelihood is constant across all parameter values by definition. However, errors in approximate inference lead to undesirable variance in the marginal likelihood estimates across different parameter values. We penalize violations of this symmetry with a textitself-consistency loss which significantly improves the quality of approximate inference in low data regimes and can be used to augment the training of popular neural density estimators. We apply our method to a number of synthetic problems and realistic scientific models, discovering notable advantages in the context of both neural posterior and likelihood approximation.