mlsanityData to 14 Oct 2026

Multivariate Conformal Selection

arXiv 2505.00917 ↗First posted 1 May 2025stat.ME cs.AI cs.LGcited by 8

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Selecting high-quality candidates from large datasets is critical in applications such as drug discovery, precision medicine, and alignment of large language models (LLMs). While Conformal Selection (CS) provides rigorous uncertainty quantification, it is limited to univariate responses and scalar criteria. To address this issue, we propose Multivariate Conformal Selection (mCS), a generalization of CS designed for multivariate response settings. Our method introduces regional monotonicity and employs multivariate nonconformity scores to construct conformal p-values, enabling finite-sample False Discovery Rate (FDR) control. We present two variants: mCS-dist, using distance-based scores, and mCS-learn, which learns optimal scores via differentiable optimization. Experiments on simulated and real-world datasets demonstrate that mCS significantly improves selection power while maintaining FDR control, establishing it as a robust framework for multivariate selection tasks.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 16 of its 45 references that are in the map. First posted on arXiv 1 May 2025.

Builds on

16 of 45 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees16 May

Other references · 15

  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  2. 2019Conformalized Quantile Regressioncited by 629
  3. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 375
  4. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
  5. 2020Fast Differentiable Sorting and Rankingcited by 197
  6. 2020Conformal Inference of Counterfactuals and Individual Treatment Effectscited by 151
  7. 2021Testing for Outliers with Conformal p-valuescited by 134
  8. 2019Differentiable Ranks and Sorting using Optimal Transportcited by 113
  9. 2019A comparison of some conformal quantile regression methodscited by 76
  10. 2022Selection by Prediction with Conformal p-valuescited by 69
  11. 2019Adaptive, Distribution-Free Prediction Intervals for Deep Networkscited by 56
  12. 2021Calibrated Multiple-Output Quantile Regression with Representation Learningcited by 50
3 more
  1. 2021Conformal Uncertainty Sets for Robust Optimizationcited by 49
  2. 2025Multivariate Conformal Prediction using Optimal Transportcited by 22
  3. 2024Semiparametric conformal predictioncited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025Multi-Condition Conformal Selectioncited by 4
  2. 2025Model-agnostic Selective Labeling with Provable Statistical Guaranteescited by 3
  3. 2025Provable Training Data Identification for Large Language Modelscited by 1
  4. 2026Beyond Fixed False Discovery Rates: Post-Hoc Conformal Selection with E-Variablescited by 1
  5. 2026Everywhere Valid Bounds on False Discovery Proportions in Conformal Inferencecited by 0
  6. 2026Provable Joint Decontamination for Benchmarking Multiple Large Language Modelscited by 0
  7. 2026Null-Calibrated Conformal Selection via Target-Membership Scorescited by 0
  8. 2026Multivariate conformal uncertainty propagation in multitask atomistic simulation: Successes and pitfallscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Robust Conformalized Selection with Noisy Responses
  2. 2024Optimized Conformal Selection: Powerful Selective Inference After Conformity Score Optimization
  3. 2025Conformal P-Value in Multiple-Choice Question Answering Tasks with Provable Risk Control
  4. 2025Diffusion-Driven High-Dimensional Variable Selection
  5. 2026Selection-Safe Conformal Early Exit: Distribution-Free Coverage under Adaptive Depth
  6. 2026Empirical Bayes Conformal Prediction for Vision and Language Models
  7. 2026Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations
  8. 2026Conformalized Large Language Models under Configuration Shift