mlsanityData to 14 Oct 2026

TECP: Token-Entropy Conformal Prediction for LLMs

arXiv 2509.00461 ↗First posted 30 Aug 2025cs.CL cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Uncertainty quantification (UQ) for open-ended language generation remains a critical yet underexplored challenge, especially under black-box constraints where internal model signals are inaccessible. In this paper, we introduce Token-Entropy Conformal Prediction (TECP), a novel framework that leverages token-level entropy as a logit-free, reference-free uncertainty measure and integrates it into a split conformal prediction (CP) pipeline to construct prediction sets with formal coverage guarantees. Unlike existing approaches that rely on semantic consistency heuristics or white-box features, TECP directly estimates epistemic uncertainty from the token entropy structure of sampled generations and calibrates uncertainty thresholds via CP quantiles to ensure provable error control. Empirical evaluations across six large language models and two benchmarks (CoQA and TriviaQA) demonstrate that TECP consistently achieves reliable coverage and compact prediction sets, outperforming prior self-consistency-based UQ methods. Our method provides a principled and efficient solution for trustworthy generation in black-box LLM settings.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 30 Aug 2025.

Builds on

22 of 34 references.

References · 22

  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  4. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  5. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  6. 2022Conformal prediction beyond exchangeabilitycited by 355
  7. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  8. 2024ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guaranteescited by 65
  9. 2024Detecting and Evaluating Medical Hallucinations in Large Vision Language Modelscited by 57
  10. 2025ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMcited by 37
  11. 2025Backdoor Cleaning without External Guidance in MLLM Fine-tuningcited by 35
  12. 2024Knowledge Graphs, Large Language Models, and Hallucinations: An NLP Perspectivecited by 29
10 more
  1. 2024CoMT: Chain-of-Medical-Thought Reduces Hallucination in Medical Report Generationcited by 25
  2. 2025COIN: Uncertainty-Guarding Selective Question Answering for Foundation Models with Provable Risk Guaranteescited by 21
  3. 2023SPOT! Revisiting Video-Language Models for Event Understandingcited by 20
  4. 2024Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Modelscited by 19
  5. 2024Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyondcited by 18
  6. 2024PediatricsGPT: Large Language Models as Chinese Medical Assistants for Pediatric Applicationscited by 16
  7. 2024Can LLMs' Tuning Methods Work in Medical Multimodal Domain?cited by 12
  8. 2025Conformal Prediction as Bayesian Quadraturecited by 6
  9. 2025Conformal Prediction with Cellwise Outliers: A Detect-then-Impute Approachcited by 5
  10. 2025MV-Debate: Multi-view Agent Debate with Dynamic Reflection Gating for Multimodal Harmful Content Detection in Social Mediacited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Entropy Alone is Insufficient for Safe Selective Prediction in LLMscited by 7
  2. 2026CAOS: Conformal Aggregation of One-Shot Predictorscited by 1
  3. 2026Easier to Judge than to Find: Predicting In-Context Learning Success for Demonstration Selectioncited by 1
  4. 2026When Entropy Is Not Enough: Reclaiming Lost Semantics in LLM Output Length Predictioncited by 0
  5. 2026Solving versus Verifying: Catching Contradictions in Tax Reasoning Systemscited by 0
  6. 2026ConformalGuard: False-Positive-Controlled Action Gating for LM Classifierscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Conformalized Large Language Models under Configuration Shift
  2. 2026Does Compression Preserve Uncertainty? A Unified Benchmark for Quantized and Sparse LLMs via Conformal Prediction
  3. 2026Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations
  4. 2026The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models
  5. 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey
  6. 2026Think Twice Before You Write -- an Entropy-based Decoding Strategy to Enhance LLM Reasoning
  7. 2025Robust Uncertainty Quantification for Self-Evolving Large Language Models via Continual Domain Pretraining
  8. 2026Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs