mlsanityData to 14 Oct 2026

Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers

arXiv 2208.11111 ↗First posted 23 Aug 2022stat.ME cs.LG math.STcited by 19

Not on the map. Why?

Abstract (arXiv)

This paper develops novel conformal methods to test whether a new observation was sampled from the same distribution as a reference set. Blending inductive and transductive conformal inference in an innovative way, the described methods can re-weight standard conformal p-values based on dependent side information from known out-of-distribution data in a principled way, and can automatically take advantage of the most powerful model from any collection of one-class and binary classifiers. The solution can be implemented either through sample splitting or via a novel transductive cross-validation+ scheme which may also be useful in other applications of conformal inference, due to tighter guarantees compared to existing cross-validation approaches. After studying false discovery rate control and power within a multiple testing framework with several possible outliers, the proposed solution is shown to outperform standard conformal p-values through simulations as well as applications to image recognition and tabular data.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 44 of its 88 references that are in the map. First posted on arXiv 23 Aug 2022.

Builds on

44 of 88 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  3. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  4. 2019Conformalized Quantile Regressioncited by 629
  5. 2014OpenML: networked science in machine learningcited by 526
  6. 2019Conformal Prediction Under Covariate Shiftcited by 519
  7. 2020Classification with Valid and Adaptive Coveragecited by 396
  8. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 366
  9. 2022Conformal prediction beyond exchangeabilitycited by 355
  10. 2022Conformal Risk Controlcited by 299
  11. 2016Least Ambiguous Set-Valued Classifiers with Bounded Error Levelscited by 272
  12. 2019Predictive inference with the jackknife+cited by 211
32 more
  1. 2018Adversarially Learned One-Class Classifier for Novelty Detectioncited by 202
  2. 2021Testing for Outliers with Conformal p-valuescited by 134
  3. 2022Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imagingcited by 107
  4. 2019Distributional conformal predictioncited by 105
  5. 2019Nested conformal prediction and quantile out-of-bag ensemble methodscited by 89
  6. 2021Conformal Prediction using Conditional Histogramscited by 77
  7. 2013One-Class Classification: Taxonomy of Study and Review of Techniquescited by 68
  8. 2019Flexible distribution-free conditional predictive bands using density estimatorscited by 64
  9. 2022Training Uncertainty-Aware Classifiers with Conformalized Deep Learningcited by 64
  10. 2019Adaptive, Distribution-Free Prediction Intervals for Deep Networkscited by 56
  11. 2020Predictive Inference Is Free with the Jackknife+-after-Bootstrapcited by 55
  12. 2022Conformal Prediction Under Feedback Covariate Shift for Biomolecular Designcited by 54
  13. 2018Effects of Degradations on Deep Neural Network Architecturescited by 48
  14. 2020Real-time Out-of-distribution Detection in Learning-Enabled Cyber-Physical Systemscited by 46
  15. 2019Prediction and outlier detection in classification problemscited by 44
  16. 2019With Malice Towards None: Assessing Uncertainty via Equalized Coveragecited by 43
  17. 2018Cautious Deep Learningcited by 35
  18. 2021A statistical framework for efficient out of distribution detection in deep neural networkscited by 28
  19. 2016Neyman-Pearson (NP) classification algorithms and NP receiver operating characteristics (NP-ROC)cited by 23
  20. 2020Training conformal predictorscited by 20
  21. 2016AdaPT: An interactive procedure for multiple testing with side informationcited by 16
  22. 2017Conformal k-NN Anomaly Detector for Univariate Data Streamscited by 13
  23. 2021Conformal Anomaly Detection on Spatio-Temporal Observations with Missing Datacited by 12
  24. 2016Multiple testing with the structure adaptive Benjamini-Hochberg algorithmcited by 11
  25. 2020Conditional calibration for false discovery rate control under dependencecited by 11
  26. 2021Semi-supervised multiple testingcited by 7
  27. 2017Covariate powered cross-weighted multiple testingcited by 6
  28. 2021A Burden Shared is a Burden Halved: A Fairness-Adjusted Approach to Classificationcited by 6
  29. 2015Neyman-Pearson Classification under High-Dimensional Settingscited by 5
  30. 2015Weighted False Discovery Rate Control in Large-Scale Multiple Testingcited by 4
  31. 2015Accumulation tests for FDR control in ordered hypothesis testingcited by 3
  32. 2022A Locally Adaptive Algorithm for Multiple Testing with Network Structurecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 4; 2025: 5; 2026: 4.

19 papers.

Sort:

By area:

  1. 2023Adaptive conformal classification with noisy labelscited by 25
  2. 2023Derandomized Novelty Detection with FDR Control via Conformal E-valuescited by 15
  3. 2023Conformal inference is (almost) free for neural networks trained with early stoppingcited by 14
  4. 2024Confidence on the Focal: Conformal Prediction with Selection-Conditional Coveragecited by 14
  5. 2024Does confidence calibration improve conformal prediction?cited by 12
  6. 2025Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularizationcited by 12
  7. 2022Conformal Frequency Estimation using Discrete Sketched Data with Coverage for Distinct Queriescited by 10
  8. 2023Conformal link prediction for false discovery rate controlcited by 9
  9. 2025Synthetic-Powered Predictive Inferencecited by 9
  10. 2023SymmPI: Predictive Inference for Data with Group Symmetriescited by 8
  11. 2024Out-of-Distribution Detection Should Use Conformal Prediction (and Vice-versa?)cited by 7
  12. 2025Membership Inference Attacks with False Discovery Rate Controlcited by 2
7 more
  1. 2024A distribution-free valid p-value for finite samples of bounded random variablescited by 1
  2. 2025On the Adversarial Robustness of Learning-based Conformal Novelty Detectioncited by 0
  3. 2025General OOD Detection via Model-aware and Subspace-aware Variable Prioritycited by 0
  4. 2026Reliable conformal novelty detection at the decision boundarycited by 0
  5. 2026ConformalHDC: Uncertainty-Aware Hyperdimensional Computing with Application to Neural Decodingcited by 0
  6. 2026Everywhere Valid Bounds on False Discovery Proportions in Conformal Inferencecited by 0
  7. 2026Conformal calibration and look-elsewhere effect in anomaly detection for new-physics searchescited by 0