mlsanityData to 14 Oct 2026

Non-Exchangeable Conformal Risk Control

arXiv 2310.01262 ↗First posted 2 Oct 2023cs.LG stat.MLcited by 29

Not on the map. Why?

Abstract (arXiv)

Split conformal prediction has recently sparked great interest due to its ability to provide formally guaranteed uncertainty sets or intervals for predictions made by black-box neural models, ensuring a predefined probability of containing the actual ground truth. While the original formulation assumes data exchangeability, some extensions handle non-exchangeable data, which is often the case in many real-world scenarios. In parallel, some progress has been made in conformal methods that provide statistical guarantees for a broader range of objectives, such as bounding the best F1-score or minimizing the false negative rate in expectation. In this paper, we leverage and extend these two lines of work by proposing non-exchangeable conformal risk control, which allows controlling the expected value of any monotone loss function when the data is not exchangeable. Our framework is flexible, makes very few assumptions, and allows weighting the data based on its relevance for a given test example; a careful choice of weights may result on tighter bounds, making our framework useful in the presence of change points, time series, or other forms of distribution drift. Experiments with both synthetic and real world data show the usefulness of our method.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 27 of its 53 references that are in the map. First posted on arXiv 2 Oct 2023.

Builds on

27 of 53 references.

References · 27

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  4. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  5. 2019Conformal Prediction Under Covariate Shiftcited by 519
  6. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 375
  7. 2022Conformal prediction beyond exchangeabilitycited by 355
  8. 2022Confident Adaptive Language Modelingcited by 310
  9. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
  10. 2022Adaptive Conformal Predictions for Time Seriescited by 161
  11. 2022Conformal Inference for Online Prediction with Arbitrary Distribution Shiftscited by 128
  12. 2013Learning by Transductioncited by 121
15 more
  1. 2021Localized Conformal Prediction: A Generalized Inference Framework for Conformal Predictioncited by 114
  2. 2019Distributional conformal predictioncited by 105
  3. 2021Distribution-free uncertainty quantification for classification under label shiftcited by 101
  4. 2023Conformal PID Control for Time Series Predictioncited by 88
  5. 2020Robust Validation: Confident Predictions Even When Distributions Shiftcited by 82
  6. 2018Exact and Robust Conformal Inference Methods for Predictive Machine Learning With Dependent Datacited by 66
  7. 2022Conformal Prediction Under Feedback Covariate Shift for Biomolecular Designcited by 54
  8. 2023How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Controlcited by 50
  9. 2021Conformal Bayesian Computationcited by 39
  10. 2022Achieving Risk Control in Online Learning Settingscited by 39
  11. 2017An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controlscited by 36
  12. 2023Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagationcited by 33
  13. 2022Copula Conformal Prediction for Multi-step Time Series Forecastingcited by 29
  14. 2023Conformal Temporal Logic Planning using Large Language Modelscited by 21
  15. 2022Conformal Prediction Intervals with Temporal Dependencecited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 7; 2025: 6; 2026: 15.

29 papers.

Sort:

By area:

  1. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 53
  2. 2023Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictionscited by 50
  3. 2024C-RAG: Certified Generation Risks for Retrieval-Augmented Language Modelscited by 27
  4. 2024Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)cited by 26
  5. 2024Conformal Prediction: A Data Perspectivecited by 25
  6. 2024Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Modelscited by 19
  7. 2025WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingalescited by 16
  8. 2024Uncertainty quantification for probabilistic machine learning in earth observation using conformal predictioncited by 9
  9. 2025Non-exchangeable Conformal Prediction with Optimal Transport: Tackling Distribution Shifts with Unlabeled Datacited by 5
  10. 2025Selective Risk Certification for LLM Outputs via Information-Lift Statistics: PAC-Bayes, Robustness, and Skeleton Designcited by 5
  11. 2026A Joint Finite-Sample Certificate for Adaptive Selective Conformal Risk Controlcited by 5
  12. 2025Calibrating Decision Robustness via Inverse Conformal Risk Controlcited by 4
17 more
  1. 2026When Can Conformal Risk Control Certify LLM Outputs? Bounds, Impossibility, and Adaptation for Structured Generationcited by 4
  2. 2026Conformal Risk Control under Non-Monotone Losses: Theory and Finite-Sample Guaranteescited by 3
  3. 2025CPATTA: Conformal Supervision Allocation For Active Test-Time Adaptationcited by 2
  4. 2026Decision-calibrated prediction sets for robust power system operationscited by 2
  5. 2024Single Trajectory Conformal Predictioncited by 1
  6. 2026Post-Hoc Trajectory-Risk Certification for Modular LLM-Based Security Agentscited by 1
  7. 2026SCP-NL2TL: Selective Conformal Prediction with Semantic Verification for Natural Language to Temporal Logic Specificationscited by 1
  8. 2025Decomposition-Based Modular Conformal Prediction for Two-Stage Modelingcited by 0
  9. 2026The Confidence Gate Theorem: When Should Ranked Decision Systems Abstain?cited by 0
  10. 2026Safe Probabilistic Planning for Human-Robot Interaction using Conformal Risk Controlcited by 0
  11. 2026Online Conformal Prediction for Non-Exchangeable Panel Datacited by 0
  12. 2026Conformal Coverage Guarantees for Any Video Temporal Groundercited by 0
  13. 2026ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learningcited by 0
  14. 2026Non-Degenerate Risk Certification for Automated Security Decisions: A Decision-Contract Theory with ATT&CK-Aligned Triage as a Worked Instancecited by 0
  15. 2026TRACE-CRC: Trajectory-Adaptive Conformal Risk Control for Multi-Step Channel State Information Predictioncited by 0
  16. 2026PixSim: a calibrated open-source simulator of instant-payment fraud, recovery and interdiction under analyst capacity constraintscited by 0
  17. 2026RAEGL: Risk-Aware Evidence-Gated Learning for Selective Contextual Routing under Temporal Shiftcited by 0