# Bayesian Quantification with Black-Box Estimators

arXiv: https://arxiv.org/abs/2302.09159 · HTML: https://mlsanity.com/paper/2302.09159/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 54 references that are in the map.
- First posted on arXiv 17 Feb 2023.

## Builds on (most cited first, 19 of 19)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [What Are Bayesian Neural Network Posteriors Really Like?](https://mlsanity.com/paper/2104.14421/) (2021)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Causality matters in medical imaging](https://mlsanity.com/paper/1912.08142/) (2019)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [A review of domain adaptation without target labels](https://mlsanity.com/paper/1901.05335/) (2019)
- [Regularized Learning for Domain Adaptation under Label Shifts](https://mlsanity.com/paper/1903.09734/) (2019)
- [A Unified View of Label Shift Estimation](https://mlsanity.com/paper/2003.07554/) (2020)
- [On Markov chain Monte Carlo methods for tall data](https://mlsanity.com/paper/1505.02827/) (2015)
- [Fisher consistency for prior probability shift](https://mlsanity.com/paper/1701.05512/) (2017)
- [Nonparametric learning from Bayesian models with randomized objective functions](https://mlsanity.com/paper/1806.11544/) (2018)
- [Quantification under prior probability shift: the ratio estimator and its extensions](https://mlsanity.com/paper/1807.03929/) (2018)
- [QuaPy: A Python-Based Framework for Quantification](https://mlsanity.com/paper/2106.11057/) (2021)
- [Approximate Models and Robust Decisions](https://mlsanity.com/paper/1402.6118/) (2014)
- [Bayesian Restricted Likelihood Methods: Conditioning on Insufficient Statistics in Bayesian Regression](https://mlsanity.com/paper/1812.07736/) (2018)
- [Confidence intervals for class prevalences under prior probability shift](https://mlsanity.com/paper/1906.04119/) (2019)
- [Unsupervised Recalibration](https://mlsanity.com/paper/1908.09157/) (2019)

## Built on by (most cited first, 4 of 4)

- [Estimating prevalence with precision and accuracy](https://mlsanity.com/paper/2507.06061/) (2025)
- [Transductive Model Selection under Prior Probability Shift](https://mlsanity.com/paper/2507.22647/) (2025)
- [Quantifying Feature Importance for Online Content Moderation](https://mlsanity.com/paper/2510.19882/) (2025)
- [Geometry-Aware Bayesian Quantification via Compositional Data Analysis](https://mlsanity.com/paper/2607.04977/) (2026)

## Abstract (arXiv)

Understanding how different classes are distributed in an unlabeled data set is an important challenge for the calibration of probabilistic classifiers and uncertainty quantification. Approaches like adjusted classify and count, black-box shift estimators, and invariant ratio estimators use an auxiliary (and potentially biased) black-box classifier trained on a different (shifted) data set to estimate the class distribution and yield asymptotic guarantees under weak assumptions. We demonstrate that all these algorithms are closely related to the inference in a particular Bayesian model, approximating the assumed ground-truth generative process. Then, we discuss an efficient Markov Chain Monte Carlo sampling scheme for the introduced model and show an asymptotic consistency guarantee in the large-data limit. We compare the introduced model against the established point estimators in a variety of scenarios, and show it is competitive, and in some cases superior, with the state of the art.