mlsanityData to 14 Oct 2026

Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference

arXiv 2310.04395 ↗First posted 6 Oct 2023cs.LG cs.AIcited by 14

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

We propose a method to improve the efficiency and accuracy of amortized Bayesian inference by leveraging universal symmetries in the joint probabilistic model of parameters and data. In a nutshell, we invert Bayes' theorem and estimate the marginal likelihood based on approximate representations of the joint model. Upon perfect approximation, the marginal likelihood is constant across all parameter values by definition. However, errors in approximate inference lead to undesirable variance in the marginal likelihood estimates across different parameter values. We penalize violations of this symmetry with a textitself-consistency loss which significantly improves the quality of approximate inference in low data regimes and can be used to augment the training of popular neural density estimators. We apply our method to a number of synthetic problems and realistic scientific models, discovering notable advantages in the context of both neural posterior and likelihood approximation.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 32 of its 51 references that are in the map. First posted on arXiv 6 Oct 2023.

Builds on

32 of 51 references.

References · 32

  1. 2022Flow Matching for Generative Modelingcited by 5,702
  2. 2015Variational Inference with Normalizing Flowscited by 2,170
  3. 2023Consistency Modelscited by 1,822
  4. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  5. 2016Variational Inference: A Review for Statisticianscited by 1,472
  6. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  7. 2019Neural Spline Flowscited by 596
  8. 2019The frontier of simulation-based inferencecited by 502
  9. 2018Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flowscited by 221
  10. 2019Automatic Posterior Transformation for Likelihood-Free Inferencecited by 219
  11. 2021Benchmarking Simulation-Based Inferencecited by 162
  12. 2020BayesFlow: Learning complex stochastic models with invertible neural networkscited by 144
20 more
  1. 2018Validating Bayesian Inference Algorithms with Simulation-Based Calibrationcited by 124
  2. 2021Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Designcited by 106
  3. 2021Real-time gravitational-wave science with neural posterior estimationcited by 78
  4. 2021Simulation Intelligence: Towards a New Generation of Scientific Methodscited by 68
  5. 2018Mining gold from implicit models to improve likelihood-free inferencecited by 60
  6. 2022Bayesian Model Selection, the Marginal Likelihood, and Generalizationcited by 57
  7. 2022Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Modelscited by 50
  8. 2022Compositional Score Modeling for Simulation-based Inferencecited by 48
  9. 2022Variational methods for simulation-based inferencecited by 43
  10. 2023JANA: Jointly Amortized Neural Approximation of Complex Bayesian Modelscited by 41
  11. 2019MadMiner: Machine learning-based inference for particle physicscited by 32
  12. 2021Sequential Neural Posterior and Likelihood Approximationcited by 30
  13. 2022Neural Importance Sampling for Rapid and Reliable Gravitational-Wave Inferencecited by 28
  14. 2021Graphical Test for Discrete Uniformity and its Applications in Goodness of Fit Evaluation and Multiple Sample Comparisoncited by 26
  15. 2018A Guide to Constraining Effective Field Theories with Machine Learningcited by 25
  16. 2020Score Matched Neural Exponential Families for Likelihood-Free Inferencecited by 21
  17. 2022Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomycited by 16
  18. 2023A Review of Change of Variable Formulas for Generative Modelingcited by 13
  19. 2020Robust model training and generalisation with Studentising flowscited by 10
  20. 2022Approximate Bayesian Computation with Domain Expert in the Loopcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 5; 2026: 5.

14 papers.

Sort:

By area:

  1. 2023Consistency Models for Scalable and Fast Simulation-Based Inferencecited by 22
  2. 2024Amortized Bayesian Multilevel Modelscited by 14
  3. 2025Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Datacited by 11
  4. 2025Multifidelity Simulation-based Inference for Computationally Expensive Simulatorscited by 9
  5. 2026BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Pythoncited by 9
  6. 2024Low-Budget Simulation-Based Inference with Bayesian Neural Networkscited by 5
  7. 2025Multilevel neural simulation-based inferencecited by 5
  8. 2025Improving the Accuracy of Amortized Model Comparison with Self-Consistencycited by 5
  9. 2024Amortized Bayesian Workflowcited by 3
  10. 2026An Introduction to Bayesian and Frequentist Simulation-Based Inference with Machine Learningcited by 1
  11. 2025Improving the Accuracy of Amortized Model Comparison with Self-Consistencycited by 0
  12. 2026Preconditioned Robust Neural Posterior Estimation for Misspecified Simulatorscited by 0
2 more
  1. 2026Unsupervised Continual Learning for Amortized Bayesian Inferencecited by 0
  2. 2026Information-Preserving Domain Transfer with Unlabeled Data in Misspecified Simulation-Based Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024All-in-one simulation-based inference
  2. 2026Amortized Factor Inference Networks for Posterior Inference
  3. 2023Adversarial robustness of amortized Bayesian inference
  4. 2024Towards Improved Variational Inference for Deep Bayesian Models
  5. 2022Amortized backward variational inference in nonlinear state-space models
  6. 2025Approximate Bayesian Inference via Bitstring Representations
  7. 2023A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors
  8. 2026Divide-and-Conquer: Towards Generalizable Amortized Bayesian Inference for the Drift Diffusion Model