# Model-free selective inference under covariate shift via weighted conformal p-values

arXiv: https://arxiv.org/abs/2307.09291 · HTML: https://mlsanity.com/paper/2307.09291/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2023.

## Built on by (most cited first, 29 of 29)

- [Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees](https://mlsanity.com/paper/2405.10301/) (2024)
- [Transductive conformal inference with adaptive scores](https://mlsanity.com/paper/2310.18108/) (2023)
- [Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration](https://mlsanity.com/paper/2406.06818/) (2024)
- [WATCH: Adaptive Monitoring for AI Deployments via Weighted-Conformal Martingales](https://mlsanity.com/paper/2505.04608/) (2025)
- [Confidence on the Focal: Conformal Prediction with Selection-Conditional Coverage](https://mlsanity.com/paper/2403.03868/) (2024)
- [Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimization](https://mlsanity.com/paper/2411.17983/) (2024)
- [Online multiple testing with e-values](https://mlsanity.com/paper/2311.06412/) (2023)
- [Conformal Selective Prediction with General Risk Control](https://mlsanity.com/paper/2603.24704/) (2026)
- [LEC: Linear Expectation Constraints for Selection-Conditioned Risk Control in Selective Prediction and Routing Systems](https://mlsanity.com/paper/2512.01556/) (2025)
- [Conformal Policy Control](https://mlsanity.com/paper/2603.02196/) (2026)
- [Conditional Testing based on Localized Conformal p-values](https://mlsanity.com/paper/2409.16829/) (2024)
- [Multi-Condition Conformal Selection](https://mlsanity.com/paper/2510.08075/) (2025)
- [Between Resolution Collapse and Variance Inflation: Weighted Conformal Anomaly Detection in Low-Data Regimes](https://mlsanity.com/paper/2603.23205/) (2026)
- [Online Conformal Selection with Accept-to-Reject Changes](https://mlsanity.com/paper/2508.13838/) (2025)
- [Model-agnostic Selective Labeling with Provable Statistical Guarantees](https://mlsanity.com/paper/2510.14581/) (2025)
- [Uncertainty Quantification With Multiple Sources](https://mlsanity.com/paper/2405.06479/) (2024)
- [Reliable algorithm selection for machine learning-guided design](https://mlsanity.com/paper/2503.20767/) (2025)
- [Causal Responder Detection](https://mlsanity.com/paper/2406.17571/) (2024)
- [SAFER: A Calibrated Risk-Aware Multimodal Recommendation Model for Dynamic Treatment Regimes](https://mlsanity.com/paper/2506.06649/) (2025)
- [Provable Training Data Identification for Large Language Models](https://mlsanity.com/paper/2510.09717/) (2025)
- [ConfHit: Conformal Generative Design with Oracle Free Guarantees](https://mlsanity.com/paper/2603.07371/) (2026)
- [Conformal Anomaly Detection in Python: Moving Beyond Heuristic Thresholds with nonconform](https://mlsanity.com/paper/2605.13642/) (2026)
- [Conformal Candidate Certification for Offline Model-Based Optimization](https://mlsanity.com/paper/2606.15217/) (2026)
- [Online Selective Conformal Prediction with Asymmetric Rules: A Permutation Test Approach](https://mlsanity.com/paper/2602.10018/) (2026)
- [Everywhere Valid Bounds on False Discovery Proportions in Conformal Inference](https://mlsanity.com/paper/2605.20726/) (2026)
- [Provable Joint Decontamination for Benchmarking Multiple Large Language Models](https://mlsanity.com/paper/2605.21543/) (2026)
- [Null-Calibrated Conformal Selection via Target-Membership Scores](https://mlsanity.com/paper/2606.22336/) (2026)
- [Certify or Refuse: A Cross-Model Map for Selective Risk Control with Coverage Floors under Covariate Shift](https://mlsanity.com/paper/2608.10893/) (2026)
- [Backward Conformal Prediction under Covariate Shift](https://mlsanity.com/paper/2610.16048/) (2026)

## Abstract (arXiv)

This paper introduces novel weighted conformal p-values and methods for model-free selective inference. The problem is as follows: given test units with covariates X and missing responses Y, how do we select units for which the responses Y are larger than user-specified values while controlling the proportion of false positives? Can we achieve this without any modeling assumptions on the data and without any restriction on the model for predicting the responses? Last, methods should be applicable when there is a covariate shift between training and test data, which commonly occurs in practice. We answer these questions by first leveraging any prediction model to produce a class of well-calibrated weighted conformal p-values, which control the type-I error in detecting a large response. These p-values cannot be passed on to classical multiple testing procedures since they may not obey a well-known positive dependence property. Hence, we introduce weighted conformalized selection (WCS), a new procedure which controls false discovery rate (FDR) in finite samples. Besides prediction-assisted candidate selection, WCS (1) allows to infer multiple individual treatment effects, and (2) extends to outlier detection with inlier distributions shifts. We demonstrate performance via simulations and applications to causal inference, drug discovery, and outlier detection datasets.