# Non-Exchangeable Conformal Risk Control

arXiv: https://arxiv.org/abs/2310.01262 · HTML: https://mlsanity.com/paper/2310.01262/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 27 of 27)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [Confident Adaptive Language Modeling](https://mlsanity.com/paper/2207.07061/) (2022)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Adaptive Conformal Predictions for Time Series](https://mlsanity.com/paper/2202.07282/) (2022)
- [Conformal Inference for Online Prediction with Arbitrary Distribution Shifts](https://mlsanity.com/paper/2208.08401/) (2022)
- [Learning by Transduction](https://mlsanity.com/paper/1301.7375/) (2013)
- [Localized Conformal Prediction: A Generalized Inference Framework for Conformal Prediction](https://mlsanity.com/paper/2106.08460/) (2021)
- [Distributional conformal prediction](https://mlsanity.com/paper/1909.07889/) (2019)
- [Distribution-free uncertainty quantification for classification under label shift](https://mlsanity.com/paper/2103.03323/) (2021)
- [Conformal PID Control for Time Series Prediction](https://mlsanity.com/paper/2307.16895/) (2023)
- [Robust Validation: Confident Predictions Even When Distributions Shift](https://mlsanity.com/paper/2008.04267/) (2020)
- [Exact and Robust Conformal Inference Methods for Predictive Machine Learning With Dependent Data](https://mlsanity.com/paper/1802.06300/) (2018)
- [Conformal Prediction Under Feedback Covariate Shift for Biomolecular Design](https://mlsanity.com/paper/2202.03613/) (2022)
- [How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Control](https://mlsanity.com/paper/2302.03791/) (2023)
- [Conformal Bayesian Computation](https://mlsanity.com/paper/2106.06137/) (2021)
- [Achieving Risk Control in Online Learning Settings](https://mlsanity.com/paper/2205.09095/) (2022)
- [An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls](https://mlsanity.com/paper/1712.09089/) (2017)
- [Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagation](https://mlsanity.com/paper/2303.12246/) (2023)
- [Copula Conformal Prediction for Multi-step Time Series Forecasting](https://mlsanity.com/paper/2212.03281/) (2022)
- [Conformal Temporal Logic Planning using Large Language Models](https://mlsanity.com/paper/2309.10092/) (2023)
- [Conformal Prediction Intervals with Temporal Dependence](https://mlsanity.com/paper/2205.12940/) (2022)

## Built on by (most cited first, 29 of 29)

- [Conformal Prediction for Natural Language Processing: A Survey](https://mlsanity.com/paper/2405.01976/) (2024)
- [Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions](https://mlsanity.com/paper/2310.05921/) (2023)
- [C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models](https://mlsanity.com/paper/2402.03181/) (2024)
- [Conformal Validity Guarantees Exist for Any Data Distribution (and How to Find Them)](https://mlsanity.com/paper/2405.06627/) (2024)
- [Conformal Prediction: A Data Perspective](https://mlsanity.com/paper/2410.06494/) (2024)
- [Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Models](https://mlsanity.com/paper/2410.08174/) (2024)
- [WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales](https://mlsanity.com/paper/2505.04608/) (2025)
- [Uncertainty quantification for probabilistic machine learning in earth observation using conformal prediction](https://mlsanity.com/paper/2401.06421/) (2024)
- [Non-exchangeable Conformal Prediction with Optimal Transport: Tackling Distribution Shifts with Unlabeled Data](https://mlsanity.com/paper/2507.10425/) (2025)
- [Selective Risk Certification for LLM Outputs via Information-Lift Statistics: PAC-Bayes, Robustness, and Skeleton Design](https://mlsanity.com/paper/2509.12527/) (2025)
- [A Joint Finite-Sample Certificate for Adaptive Selective Conformal Risk Control](https://mlsanity.com/paper/2606.08517/) (2026)
- [Calibrating Decision Robustness via Inverse Conformal Risk Control](https://mlsanity.com/paper/2510.07750/) (2025)
- [When Can Conformal Risk Control Certify LLM Outputs? Bounds, Impossibility, and Adaptation for Structured Generation](https://mlsanity.com/paper/2606.29054/) (2026)
- [Conformal Risk Control under Non-Monotone Losses: Theory and Finite-Sample Guarantees](https://mlsanity.com/paper/2604.01502/) (2026)
- [CPATTA: Conformal Supervision Allocation For Active Test-Time Adaptation](https://mlsanity.com/paper/2509.25692/) (2025)
- [Decision-calibrated prediction sets for robust power system operations](https://mlsanity.com/paper/2606.02081/) (2026)
- [Single Trajectory Conformal Prediction](https://mlsanity.com/paper/2406.01570/) (2024)
- [Post-Hoc Trajectory-Risk Certification for Modular LLM-Based Security Agents](https://mlsanity.com/paper/2608.05199/) (2026)
- [SCP-NL2TL: Selective Conformal Prediction with Semantic Verification for Natural Language to Temporal Logic Specifications](https://mlsanity.com/paper/2608.05439/) (2026)
- [Decomposition-Based Modular Conformal Prediction for Two-Stage Modeling](https://mlsanity.com/paper/2510.04406/) (2025)
- [The Confidence Gate Theorem: When Should Ranked Decision Systems Abstain?](https://mlsanity.com/paper/2603.09947/) (2026)
- [Safe Probabilistic Planning for Human-Robot Interaction using Conformal Risk Control](https://mlsanity.com/paper/2603.10392/) (2026)
- [Online Conformal Prediction for Non-Exchangeable Panel Data](https://mlsanity.com/paper/2605.17705/) (2026)
- [Conformal Coverage Guarantees for Any Video Temporal Grounder](https://mlsanity.com/paper/2608.07434/) (2026)
- [ReliableNet: A Chance-Constrained Approach to Trustworthy Classification in Deep Learning](https://mlsanity.com/paper/2608.09768/) (2026)
- [Non-Degenerate Risk Certification for Automated Security Decisions: A Decision-Contract Theory with ATT&CK-Aligned Triage as a Worked Instance](https://mlsanity.com/paper/2608.12444/) (2026)
- [TRACE-CRC: Trajectory-Adaptive Conformal Risk Control for Multi-Step Channel State Information Prediction](https://mlsanity.com/paper/2608.27124/) (2026)
- [PixSim: a calibrated open-source simulator of instant-payment fraud, recovery and interdiction under analyst capacity constraints](https://mlsanity.com/paper/2609.30684/) (2026)
- [RAEGL: Risk-Aware Evidence-Gated Learning for Selective Contextual Routing under Temporal Shift](https://mlsanity.com/paper/2609.33340/) (2026)

## Abstract (arXiv)

Split conformal prediction has recently sparked great interest due to its ability to provide formally guaranteed uncertainty sets or intervals for predictions made by black-box neural models, ensuring a predefined probability of containing the actual ground truth. While the original formulation assumes data exchangeability, some extensions handle non-exchangeable data, which is often the case in many real-world scenarios. In parallel, some progress has been made in conformal methods that provide statistical guarantees for a broader range of objectives, such as bounding the best F1-score or minimizing the false negative rate in expectation. In this paper, we leverage and extend these two lines of work by proposing non-exchangeable conformal risk control, which allows controlling the expected value of any monotone loss function when the data is not exchangeable. Our framework is flexible, makes very few assumptions, and allows weighting the data based on its relevance for a given test example; a careful choice of weights may result on tighter bounds, making our framework useful in the presence of change points, time series, or other forms of distribution drift. Experiments with both synthetic and real world data show the usefulness of our method.