mlsanityData to 7 Oct 2026

Adapting Prediction Sets to Distribution Shifts Without Labels

arXiv 2406.01416 ↗First posted 3 Jun 2024cs.LG stat.MLcited by 6

Not on the map. Why?

Abstract (arXiv)

Recently there has been a surge of interest to deploy confidence set predictions rather than point predictions in machine learning. Unfortunately, the effectiveness of such prediction sets is frequently impaired by distribution shifts in practice, and the challenge is often compounded by the lack of ground truth labels at test time. Focusing on a standard set-valued prediction framework called conformal prediction (CP), this paper studies how to improve its practical performance using only unlabeled data from the shifted test domain. This is achieved by two new methods called ECP and EACP, whose main idea is to adjust the score function in CP according to its base model's own uncertainty evaluation. Through extensive experiments on a number of large-scale datasets and neural network architectures, we show that our methods provide consistent improvement over existing baselines and nearly match the performance of fully supervised methods.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 47 of its 63 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

47 of 63 references.

References · 47

  1. 2016Densely Connected Convolutional Networkscited by 6,691
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  5. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  6. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  7. 2019Natural Adversarial Examplescited by 1,401
  8. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  9. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  10. 2019Conformalized Quantile Regressioncited by 620
  11. 2019Conformal Prediction Under Covariate Shiftcited by 511
  12. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
35 more
  1. 2020Improving robustness against common corruptions by covariate shift adaptationcited by 427
  2. 2020Classification with Valid and Adaptive Coveragecited by 390
  3. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  4. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  5. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  6. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 360
  7. 2022Conformal prediction beyond exchangeabilitycited by 348
  8. 2017Functional Map of the Worldcited by 338
  9. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
  10. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 270
  11. 2020Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shiftcited by 235
  12. 2023Conformal Language Modelingcited by 163
  13. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 131
  14. 2024Language Models with Conformal Factuality Guaranteescited by 128
  15. 2022Conformal Inference for Online Prediction with Arbitrary Distribution Shiftscited by 123
  16. 2023EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularizationcited by 123
  17. 2022Safe Planning in Dynamic Environments using Conformal Predictioncited by 112
  18. 2020The iWildCam 2020 Competition Datasetcited by 86
  19. 2023Conformal PID Control for Time Series Predictioncited by 85
  20. 2023Improved Online Conformal Prediction via Strongly Adaptive Online Learningcited by 76
  21. 2022Practical Adversarial Multivalid Conformal Predictioncited by 65
  22. 2019Flexible distribution-free conditional predictive bands using density estimatorscited by 64
  23. 2021SITA: Single Image Test-time Adaptationcited by 57
  24. 2023RDumb: A simple approach that questions our progress in continual test-time adaptationcited by 50
  25. 2024Conformal Alignment: Knowing When to Trust Foundation Models with Guaranteescited by 50
  26. 2023Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictionscited by 46
  27. 2023Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagationcited by 33
  28. 2021Set-valued classification -- overview via a unified frameworkcited by 27
  29. 2024The Entropy Enigma: Success and Failure of Entropy Minimizationcited by 24
  30. 2023Improving Adaptive Conformal Prediction Using Self-Supervised Learningcited by 17
  31. 2023Integrating Uncertainty Awareness into Conformalized Quantile Regressioncited by 17
  32. 2024Not all distributional shifts are equal: Fine-grained robust conformal inferencecited by 15
  33. 2023Empirically Validating Conformal Prediction on Modern Vision Architectures Under Distribution Shift and Long-tailed Datacited by 7
  34. 2024CLOSURE: Fast Quantification of Pose Uncertainty Setscited by 6
  35. 2024Optimal Aggregation of Prediction Intervals under Unsupervised Domain Shiftcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Coverage Guarantees for Pseudo-Calibrated Conformal Prediction under Distribution Shiftcited by 1
  2. 2026Retrospective Counterfactual Prediction by Conditioning on the Factual Outcome: A Cross-World Approachcited by 1
  3. 2026Diagnosing Conformal Prediction Failures Under Distribution Shift: A COVID-19 Case Studycited by 0
  4. 2026DistMatch: Adaptive Binning via Distribution Matching for Robust Sequential Conformal Predictioncited by 0
  5. 2026The Label Complexity of Useful Class-Conditional Prediction Sets under Distribution Shiftcited by 0
  6. 2026Does Marginal Coverage Guarantee Class-Conditional Safety for Zero-Shot VLMs Under Shift?cited by 0