mlsanityData to 7 Oct 2026

A Unified View of Label Shift Estimation

arXiv 2003.07554 ↗First posted 17 Mar 2020cs.LG stat.MLcited by 119

Not on the map. Why?

Abstract (arXiv)

Under label shift, the label distribution p(y) might change but the class-conditional distributions p(x|y) do not. There are two dominant approaches for estimating the label marginal. BBSE, a moment-matching approach based on confusion matrices, is provably consistent and provides interpretable error bounds. However, a maximum likelihood estimation approach, which we call MLLS, dominates empirically. In this paper, we present a unified view of the two methods and the first theoretical characterization of MLLS. Our contributions include (i) consistency conditions for MLLS, which include calibration of the classifier and a confusion matrix invertibility condition that BBSE also requires; (ii) a unified framework, casting BBSE as roughly equivalent to MLLS for a particular choice of calibration method; and (iii) a decomposition of MLLS's finite-sample error into terms reflecting miscalibration and estimation error. Our analysis attributes BBSE's statistical inefficiency to a loss of information due to coarse calibration. Experiments on synthetic data, MNIST, and CIFAR10 support our findings.

Key facts. Cited by 119 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 19; 2023: 28; 2024: 17; 2025: 21; 2026: 18.

119 papers.

Sort:

By area:

  1. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  2. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  3. 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performancecited by 141
  4. 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 112
  5. 2021Distribution-free uncertainty quantification for classification under label shiftcited by 99
  6. 2021Mixture Proportion Estimation and PU Learning: A Modern Approachcited by 52
  7. 2022Diagnosing failures of fairness transfer across distribution shift in real-world medical settingscited by 45
  8. 2021Online Adaptation to Label Distribution Shiftcited by 44
  9. 2023Subspace Identification for Multi-Source Domain Adaptationcited by 36
  10. 2022Domain Adaptation under Open Set Label Shiftcited by 32
  11. 2023RLSbench: Domain Adaptation Under Relaxed Label Shiftcited by 30
  12. 2023Conformal Prediction for Federated Uncertainty Quantification Under Label Shiftcited by 29
38 more
  1. 2022Adapting to Online Label Shift with Provable Guaranteescited by 26
  2. 2021Aggregating From Multiple Target-Shifted Sourcescited by 24
  3. 2023Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimationcited by 22
  4. 2022Deconstructing Distributions: A Pointwise Framework of Learningcited by 19
  5. 2023PAC Prediction Sets Under Label Shiftcited by 19
  6. 2021Mapping conditional distributions for domain adaptation under generalized target shiftcited by 18
  7. 2021A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalizationcited by 18
  8. 2023GeoNet: Benchmarking Unsupervised Adaptation across Geographiescited by 17
  9. 2021Amazon SageMaker Model Monitor: A System for Real-Time Insights into Deployed Machine Learning Modelscited by 16
  10. 2023Adapting to Continuous Covariate Shift via Online Density Ratio Estimationcited by 15
  11. 2021MLDemon: Deployment Monitoring for Machine Learning Systemscited by 14
  12. 2021Contrastive Attraction and Contrastive Repulsion for Representation Learningcited by 13
  13. 2022Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Surveycited by 12
  14. 2022Adapting to Latent Subgroup Shifts via Concepts and Proxiescited by 12
  15. 2022Understanding new tasks through the lens of training data via exponential tiltingcited by 11
  16. 2022Domain Adaptation under Missingness Shiftcited by 11
  17. 2023Online Label Shift: Optimal Dynamic Regret meets Practical Algorithmscited by 11
  18. 2023Adaptive Training Distributions with Scalable Online Bilevel Optimizationcited by 11
  19. 2023Minimum-Risk Recalibration of Classifierscited by 10
  20. 2024Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognitioncited by 10
  21. 2022A Human-Centric Take on Model Monitoringcited by 9
  22. 2022Unsupervised Learning under Latent Label Shiftcited by 9
  23. 2023ELSA: Efficient Label Shift Adaptation through the Lens of Semiparametric Modelscited by 9
  24. 2023Kernel Density Estimation for Multiclass Quantificationcited by 8
  25. 2025Robust Learning of Multi-index Models via Iterative Subspace Approximationcited by 8
  26. 2025When to retrain a machine learning modelcited by 8
  27. 2025Statistically Valid Post-Deployment Monitoring Should Be Standard for AI-Based Digital Healthcited by 8
  28. 2022Beyond Invariance: Test-Time Label-Shift Adaptation for Distributions with "Spurious" Correlationscited by 7
  29. 2023Bi-directional Distribution Alignment for Transductive Zero-Shot Learningcited by 7
  30. 2022Importance Tempering: Group Robustness for Overparameterized Modelscited by 6
  31. 2022Explanation Shift: Detecting distribution shifts on tabular data via the explanation spacecited by 6
  32. 2024FineFake: A Knowledge-Enriched Dataset for Fine-Grained Multi-Domain Fake News Detectioncited by 6
  33. 2024What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?cited by 6
  34. 2024UDA-Bench: Revisiting Common Assumptions in Unsupervised Domain Adaptation Using a Standardized Frameworkcited by 6
  35. 2021Understanding and Testing Generalization of Deep Networks on Out-of-Distribution Datacited by 5
  36. 2022Time-Varying Propensity Score to Bridge the Gap between the Past and Presentcited by 5
  37. 2023Explanation Shift: How Did the Distribution Shift Impact the Model?cited by 5
  38. 2023Label Shift Quantification with Robustness Guarantees via Distribution Feature Matchingcited by 5

50 most cited of 119 All 119 (JSON)