# Improving the Accuracy of Amortized Model Comparison with Self-Consistency

arXiv: https://arxiv.org/abs/2508.20614 · HTML: https://mlsanity.com/paper/2508.20614/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 25 of 25)

- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [BayesFlow: Learning complex stochastic models with invertible neural networks](https://mlsanity.com/paper/2003.06281/) (2020)
- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [Detecting Model Misspecification in Amortized Bayesian Inference with Neural Networks](https://mlsanity.com/paper/2112.08866/) (2021)
- [JANA: Jointly Amortized Neural Approximation of Complex Bayesian Models](https://mlsanity.com/paper/2302.09125/) (2023)
- [Marginal likelihood computation for model selection and hypothesis testing: an extensive review](https://mlsanity.com/paper/2005.08334/) (2020)
- [BayesFlow: Amortized Bayesian Workflows With Neural Networks](https://mlsanity.com/paper/2306.16015/) (2023)
- [Amortized Bayesian model comparison with evidential deep learning](https://mlsanity.com/paper/2004.10629/) (2020)
- [Sensitivity-Aware Amortized Bayesian Inference](https://mlsanity.com/paper/2310.11122/) (2023)
- [On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows](https://mlsanity.com/paper/2402.06578/) (2024)
- [The Statistical Accuracy of Neural Posterior and Likelihood Estimation](https://mlsanity.com/paper/2411.12068/) (2024)
- [Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference](https://mlsanity.com/paper/2310.04395/) (2023)
- [Amortized Bayesian Multilevel Models](https://mlsanity.com/paper/2408.13230/) (2024)
- [Bayesian Evidence and Model Selection](https://mlsanity.com/paper/1411.3013/) (2014)
- [A Deep Learning Method for Comparing Bayesian Hierarchical Models](https://mlsanity.com/paper/2301.11873/) (2023)
- [Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data](https://mlsanity.com/paper/2501.13483/) (2025)
- [Evidence Networks: simple losses for fast, amortized, neural Bayesian model comparison](https://mlsanity.com/paper/2305.11241/) (2023)
- [Simultaneous identification of models and parameters of scientific simulators](https://mlsanity.com/paper/2305.15174/) (2023)
- [Bayesian evidence estimation from posterior samples with normalizing flows](https://mlsanity.com/paper/2404.12294/) (2024)
- [Validation and Comparison of Non-Stationary Cognitive Models: A Diffusion Model Application](https://mlsanity.com/paper/2401.08626/) (2023)

## Built on by (most cited first, 5 of 5)

- [BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Python](https://mlsanity.com/paper/2602.07098/) (2026)
- [Scalable Simulation-Based Model Inference with Test-Time Complexity Control](https://mlsanity.com/paper/2603.15292/) (2026)
- [Improving the Accuracy of Amortized Model Comparison with Self-Consistency](https://mlsanity.com/paper/2512.14308/) (2025)
- [Unsupervised Continual Learning for Amortized Bayesian Inference](https://mlsanity.com/paper/2602.22884/) (2026)
- [Identifiability and amortized inference limitations in Kuramoto models](https://mlsanity.com/paper/2603.21752/) (2026)

## Abstract (arXiv)

Amortized Bayesian model comparison (BMC) enables fast probabilistic ranking of models via simulation-based training of neural surrogates. However, the accuracy of neural surrogates deteriorates when simulation models are misspecified; the very case where model comparison is most needed. We evaluate four different amortized BMC methods. We supplement traditional simulation-based training of these methods with a self-consistency (SC) loss on unlabeled real data to improve BMC estimates under distribution shifts. Using one artificial and two real-world case studies, we compare amortized BMC estimators with and without SC against analytic or bridge sampling benchmarks. In the closed-world case (data is generated by one of the candidate models), BMC estimators using classifiers work acceptably well even without SC training. However, these methods also benefit the least from SC training. In the open-world scenario (all models misspecified), SC training strongly improves BMC estimators when having access to analytic likelihoods, or when surrogate likelihoods are locally accurate near the true parameter posterior, even for severely misspecified models. We conclude with practical recommendations for amortized BMC and suggestions for future research.