# SConU: Selective Conformal Uncertainty in Large Language Models

arXiv: https://arxiv.org/abs/2504.14154 · HTML: https://mlsanity.com/paper/2504.14154/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2025.

## Built on by (most cited first, 23 of 23)

- [From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered](https://mlsanity.com/paper/2506.07461/) (2025)
- [Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards](https://mlsanity.com/paper/2603.09117/) (2026)
- [Conformal Information Pursuit for Interactively Guiding Large Language Models](https://mlsanity.com/paper/2507.03279/) (2025)
- [Do Large Language Models (Really) Need Statistical Foundations?](https://mlsanity.com/paper/2505.19145/) (2025)
- [FESTA: Functionally Equivalent Sampling for Trust Assessment of Multimodal LLMs](https://mlsanity.com/paper/2509.16648/) (2025)
- [Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review](https://mlsanity.com/paper/2504.18346/) (2025)
- [Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Prediction](https://mlsanity.com/paper/2504.17671/) (2025)
- [Conformal Segmentation in Industrial Surface Defect Detection with Statistical Guarantees](https://mlsanity.com/paper/2504.17721/) (2025)
- [Conformal Sets in Multiple-Choice Question Answering under Black-Box Settings with Provable Coverage Guarantees](https://mlsanity.com/paper/2508.05544/) (2025)
- [MM-Snowball: Evaluating and Mitigating Hallucination Snowballing in Multimodal Multi-Turn Dialogue](https://mlsanity.com/paper/2606.00622/) (2026)
- [Judge, Retrieve, or Abstain: Uncertainty-Guarded LLM Judging with Provable Risk Guarantees](https://mlsanity.com/paper/2608.17994/) (2026)
- [Credal Large Language Models for Semantic Commitment under Uncertainty](https://mlsanity.com/paper/2608.23244/) (2026)
- [Conformal Prediction for Indoor Positioning with Correctness Coverage Guarantees](https://mlsanity.com/paper/2505.01810/) (2025)
- [Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations](https://mlsanity.com/paper/2604.16217/) (2026)
- [Uncertainty-Aware Abstention in Large Language Models with Provable Alignment Guarantees](https://mlsanity.com/paper/2607.04430/) (2026)
- [SCP-NL2TL: Selective Conformal Prediction with Semantic Verification for Natural Language to Temporal Logic Specifications](https://mlsanity.com/paper/2608.05439/) (2026)
- [Calibrated Prediction Set in Fault Detection with Risk Guarantees via Significance Tests](https://mlsanity.com/paper/2508.01208/) (2025)
- [Conformal P-Value in Multiple-Choice Question Answering Tasks with Provable Risk Control](https://mlsanity.com/paper/2508.10022/) (2025)
- [Hierarchical Spatio-Temporal Attention Network with Adaptive Risk-Aware Decision for Forward Collision Warning in Complex Scenarios](https://mlsanity.com/paper/2511.19952/) (2025)
- [Beyond Model Jailbreak: Systematic Dissection of the "Ten DeadlySins" in Embodied Intelligence](https://mlsanity.com/paper/2512.06387/) (2025)
- [Occupancy-based Quantile Risk Control](https://mlsanity.com/paper/2609.03104/) (2026)
- [Can VLMs Reliably Assess Sidewalk Accessibility Attributes from Pedestrian-Level Imagery?](https://mlsanity.com/paper/2609.17882/) (2026)
- [Unanimously Wrong: Certified Abstention from How Medical LLM Consensus Forms](https://mlsanity.com/paper/2610.07570/) (2026)

## Abstract (arXiv)

As large language models are increasingly utilized in real-world applications, guarantees of task-specific metrics are essential for their reliable deployment. Previous studies have introduced various criteria of conformal uncertainty grounded in split conformal prediction, which offer user-specified correctness coverage. However, existing frameworks often fail to identify uncertainty data outliers that violate the exchangeability assumption, leading to unbounded miscoverage rates and unactionable prediction sets. In this paper, we propose a novel approach termed Selective Conformal Uncertainty (SConU), which, for the first time, implements significance tests, by developing two conformal p-values that are instrumental in determining whether a given sample deviates from the uncertainty distribution of the calibration set at a specific manageable risk level. Our approach not only facilitates rigorous management of miscoverage rates across both single-domain and interdisciplinary contexts, but also enhances the efficiency of predictions. Furthermore, we comprehensively analyze the components of the conformal procedures, aiming to approximate conditional coverage, particularly in high-stakes question-answering tasks.