# Classification under Nuisance Parameters and Generalized Label Shift in Likelihood-Free Inference

arXiv: https://arxiv.org/abs/2402.05330 · HTML: https://mlsanity.com/paper/2402.05330/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 51 references that are in the map.
- First posted on arXiv 8 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [Masked Autoregressive Flow for Density Estimation](https://mlsanity.com/paper/1705.07057/) (2017)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Detecting and Correcting for Label Shift with Black Box Predictors](https://mlsanity.com/paper/1802.03916/) (2018)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Distribution-free uncertainty quantification for classification under label shift](https://mlsanity.com/paper/2103.03323/) (2021)
- [A Living Review of Machine Learning for Particle Physics](https://mlsanity.com/paper/2102.02770/) (2021)
- [Mining gold from implicit models to improve likelihood-free inference](https://mlsanity.com/paper/1805.12244/) (2018)
- [Learning to Pivot with Adversarial Networks](https://mlsanity.com/paper/1611.01046/) (2016)
- [Criteria of efficiency for conformal prediction](https://mlsanity.com/paper/1603.04416/) (2016)
- [Diagnostics for Conditional Density Models and Bayesian Inference Algorithms](https://mlsanity.com/paper/2102.10473/) (2021)
- [Quantification under prior probability shift: the ratio estimator and its extensions](https://mlsanity.com/paper/1807.03929/) (2018)
- [Simulator-Based Inference with Waldo: Confidence Regions by Leveraging Prediction Algorithms and Posterior Estimators for Inverse Problems](https://mlsanity.com/paper/2205.15680/) (2022)
- [Learning Likelihood Ratios with Neural Network Classifiers](https://mlsanity.com/paper/2305.10500/) (2023)
- [Dealing with Nuisance Parameters using Machine Learning in High Energy Physics: a Review](https://mlsanity.com/paper/2007.09121/) (2020)
- [A unified framework for dataset shift diagnostics](https://mlsanity.com/paper/2205.08340/) (2022)
- [From conformal to probabilistic prediction](https://mlsanity.com/paper/1406.5600/) (2014)

## Built on by (most cited first, 1 of 1)

- [From matrix inversion to constraints: provably tighter confidence regions for importance weights in label shift](https://mlsanity.com/paper/2609.14802/) (2026)

## Abstract (arXiv)

An open scientific challenge is how to classify events with reliable measures of uncertainty, when we have a mechanistic model of the data-generating process but the distribution over both labels and latent nuisance parameters is different between train and target data. We refer to this type of distributional shift as generalized label shift (GLS). Direct classification using observed data X as covariates leads to biased predictions and invalid uncertainty estimates of labels Y. We overcome these biases by proposing a new method for robust uncertainty quantification that casts classification as a hypothesis testing problem under nuisance parameters. The key idea is to estimate the classifier's receiver operating characteristic (ROC) across the entire nuisance parameter space, which allows us to devise cutoffs that are invariant under GLS. Our method effectively endows a pre-trained classifier with domain adaptation capabilities and returns valid prediction sets while maintaining high power. We demonstrate its performance on two challenging scientific problems in biology and astroparticle physics with data from realistic mechanistic models.