mlsanityData to 7 Oct 2026

Practical Adversarial Multivalid Conformal Prediction

arXiv 2206.01067 ↗First posted 2 Jun 2022cs.LGcited by 65

Not on the map. Why?

Abstract (arXiv)

We give a simple, generic conformal prediction method for sequential prediction that achieves target empirical coverage guarantees against adversarially chosen data. It is computationally lightweight -- comparable to split conformal prediction -- but does not require having a held-out validation set, and so all data can be used for training models from which to derive a conformal score. It gives stronger than marginal coverage guarantees in two ways. First, it gives threshold calibrated prediction sets that have correct empirical coverage even conditional on the threshold used to form the prediction set from the conformal score. Second, the user can specify an arbitrary collection of subsets of the feature space -- possibly intersecting -- and the coverage guarantees also hold conditional on membership in each of these subsets. We call our algorithm MVP, short for MultiValid Prediction. We give both theory and an extensive set of empirical evaluations.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

19 of 31 references.

References · 19

  1. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  2. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  3. 2019Conformalized Quantile Regressioncited by 620
  4. 2019Conformal Prediction Under Covariate Shiftcited by 511
  5. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
  6. 2020Classification with Valid and Adaptive Coveragecited by 390
  7. 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 381
  8. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  9. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 360
  10. 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 221
  11. 2022Adaptive Conformal Predictions for Time Seriescited by 155
  12. 2018An Empirical Study of Rich Subgroup Fairness for Machine Learningcited by 81
7 more
  1. 2018Exact and Robust Conformal Inference Methods for Predictive Machine Learning With Dependent Datacited by 64
  2. 2020Moment Multicalibration for Uncertainty Estimationcited by 63
  3. 2019PAC Confidence Sets for Deep Neural Networks via Calibrated Predictioncited by 59
  4. 2018Fairness Through Computationally-Bounded Awarenesscited by 55
  5. 2021Online Multivalid Learning: Means, Moments, and Prediction Intervalscited by 48
  6. 2019With Malice Towards None: Assessing Uncertainty via Equalized Coveragecited by 43
  7. 2021Bayes-optimal prediction with frequentist coverage controlcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 11; 2024: 20; 2025: 15; 2026: 12.

65 papers.

Sort:

By area:

  1. 2022Conformal Inference for Online Prediction with Arbitrary Distribution Shiftscited by 123
  2. 2023Conformal PID Control for Time Series Predictioncited by 85
  3. 2022Batch Multivalid Conformal Predictioncited by 66
  4. 2023Scalable Membership Inference Attacks via Quantile Regressioncited by 61
  5. 2024Online conformal prediction with decaying step sizescited by 53
  6. 2023Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictionscited by 46
  7. 2024Multicalibration for Confidence Scoring in LLMscited by 44
  8. 2023High-Dimensional Prediction for Sequential Decision Makingcited by 41
  9. 2025Decision Theoretic Foundations for Conformal Prediction: Optimal Uncertainty Quantification for Risk-Averse Agentscited by 37
  10. 2023Oracle Efficient Online Multicalibration and Omnipredictioncited by 36
  11. 2023A Unifying Perspective on Multi-Calibration: Game Dynamics for Multi-Objective Learningcited by 29
  12. 2022Distribution Free Prediction Sets for Node Classificationcited by 28
38 more
  1. 2024Provably Robust Conformal Prediction with Improved Efficiencycited by 27
  2. 2023Beyond Confidence: Reliable Models Should Also Consider Atypicalitycited by 25
  3. 2024Conformal Prediction: A Data Perspectivecited by 25
  4. 2024On the Distance from Calibration in Sequential Predictioncited by 18
  5. 2023HappyMap: A Generalized Multi-calibration Methodcited by 17
  6. 2023Equal Opportunity of Coverage in Fair Regressioncited by 16
  7. 2024Self-Calibrating Conformal Predictioncited by 16
  8. 2024On Computationally Efficient Multi-Class Calibrationcited by 16
  9. 2024Localized Adaptive Risk Controlcited by 16
  10. 2024When is Multicalibration Post-Processing Necessary?cited by 16
  11. 2024Automatically Adaptive Conformal Risk Controlcited by 16
  12. 2022Reconciling Individual Probability Forecastscited by 14
  13. 2024Fair Risk Control: A Generalized Framework for Calibrating Multi-group Fairness Riskscited by 13
  14. 2025The Relationship between No-Regret Learning and Online Conformal Predictioncited by 13
  15. 2024Discounted Adaptive Online Learning: Towards Better Regularizationcited by 12
  16. 2024Split Conformal Prediction under Data Contaminationcited by 12
  17. 2022Conformal Methods for Quantifying Uncertainty in Spatiotemporal Data: A Surveycited by 11
  18. 2025Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularizationcited by 11
  19. 2025Conformal Prediction under Levy-Prokhorov Distribution Shifts: Robustness to Local and Global Perturbationscited by 11
  20. 2024Human-AI Safety: A Descendant of Generative AI and Control Systems Safetycited by 7
  21. 2024Stochastic Online Conformal Prediction with Semi-Bandit Feedbackcited by 7
  22. 2025A Generic Framework for Conformal Fairnesscited by 7
  23. 2023The Scope of Multicalibration: Characterizing Multicalibration via Property Elicitationcited by 6
  24. 2024Adapting Prediction Sets to Distribution Shifts Without Labelscited by 6
  25. 2023Conformal Policy Learning for Sensorimotor Control Under Distribution Shiftscited by 5
  26. 2025In Defense of Defensive Forecastingcited by 5
  27. 2026Conformal Risk Control for Non-Monotonic Lossescited by 5
  28. 2026Conformal Policy Controlcited by 5
  29. 2024Stability and Multigroup Fairness in Ranking with Uncertain Predictionscited by 4
  30. 2024Calibrated Probabilistic Forecasts for Arbitrary Sequencescited by 4
  31. 2024Adaptive Conformal Inference by Particle Filtering under Hidden Markov Modelscited by 4
  32. 2025Improved Bounds for Swap Multicalibration and Swap Omnipredictioncited by 4
  33. 2025Conformal Prediction for Time-series Forecasting with Change Pointscited by 4
  34. 2025Efficient Swap Multicalibration of Elicitable Propertiescited by 4
  35. 2026Online Conformal Prediction with Adversarial Semi-bandit Feedback via Regret Minimizationcited by 4
  36. 2022Context matters for fairness -- a case study on the effect of spatial distribution shiftscited by 3
  37. 2024The Benefit of Being Bayesian in Online Conformal Predictioncited by 3
  38. 2022ACon2: Adaptive Conformal Consensus for Provable Blockchain Oraclescited by 2

50 most cited of 65 All 65 (JSON)