# Multivariate Conformal Selection

arXiv: https://arxiv.org/abs/2505.00917 · HTML: https://mlsanity.com/paper/2505.00917/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 16 of its 45 references that are in the map.
- First posted on arXiv 1 May 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)

## Builds on (most cited first, 16 of 16)

- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Analyzing Learned Molecular Representations for Property Prediction](https://mlsanity.com/paper/1904.01561/) (2019)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [Conformal Inference of Counterfactuals and Individual Treatment Effects](https://mlsanity.com/paper/2006.06138/) (2020)
- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Differentiable Ranks and Sorting using Optimal Transport](https://mlsanity.com/paper/1905.11885/) (2019)
- [A comparison of some conformal quantile regression methods](https://mlsanity.com/paper/1909.05433/) (2019)
- [Selection by Prediction with Conformal p-values](https://mlsanity.com/paper/2210.01408/) (2022)
- [Adaptive, Distribution-Free Prediction Intervals for Deep Networks](https://mlsanity.com/paper/1905.10634/) (2019)
- [Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees](https://mlsanity.com/paper/2405.10301/) (2024)
- [Calibrated Multiple-Output Quantile Regression with Representation Learning](https://mlsanity.com/paper/2110.00816/) (2021)
- [Conformal Uncertainty Sets for Robust Optimization](https://mlsanity.com/paper/2105.14957/) (2021)
- [Multivariate Conformal Prediction using Optimal Transport](https://mlsanity.com/paper/2502.03609/) (2025)
- [Semiparametric conformal prediction](https://mlsanity.com/paper/2411.02114/) (2024)

## Built on by (most cited first, 8 of 8)

- [Multi-Condition Conformal Selection](https://mlsanity.com/paper/2510.08075/) (2025)
- [Model-agnostic Selective Labeling with Provable Statistical Guarantees](https://mlsanity.com/paper/2510.14581/) (2025)
- [Provable Training Data Identification for Large Language Models](https://mlsanity.com/paper/2510.09717/) (2025)
- [Beyond Fixed False Discovery Rates: Post-Hoc Conformal Selection with E-Variables](https://mlsanity.com/paper/2604.11305/) (2026)
- [Everywhere Valid Bounds on False Discovery Proportions in Conformal Inference](https://mlsanity.com/paper/2605.20726/) (2026)
- [Provable Joint Decontamination for Benchmarking Multiple Large Language Models](https://mlsanity.com/paper/2605.21543/) (2026)
- [Null-Calibrated Conformal Selection via Target-Membership Scores](https://mlsanity.com/paper/2606.22336/) (2026)
- [Multivariate conformal uncertainty propagation in multitask atomistic simulation: Successes and pitfalls](https://mlsanity.com/paper/2609.31384/) (2026)

## Abstract (arXiv)

Selecting high-quality candidates from large datasets is critical in applications such as drug discovery, precision medicine, and alignment of large language models (LLMs). While Conformal Selection (CS) provides rigorous uncertainty quantification, it is limited to univariate responses and scalar criteria. To address this issue, we propose Multivariate Conformal Selection (mCS), a generalization of CS designed for multivariate response settings. Our method introduces regional monotonicity and employs multivariate nonconformity scores to construct conformal p-values, enabling finite-sample False Discovery Rate (FDR) control. We present two variants: mCS-dist, using distance-based scores, and mCS-learn, which learns optimal scores via differentiable optimization. Experiments on simulated and real-world datasets demonstrate that mCS significantly improves selection power while maintaining FDR control, establishing it as a robust framework for multivariate selection tasks.