mlsanityData to 13 Oct 2026

Conformal Nucleus Sampling

arXiv 2305.02633 ↗First posted 4 May 2023cs.CL cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Language models generate text based on successively sampling the next word. A decoding procedure based on nucleus (top-p) sampling chooses from the smallest possible set of words whose cumulative probability exceeds the probability p. In this work, we assess whether a top-p set is indeed aligned with its probabilistic meaning in various linguistic contexts. We employ conformal prediction, a calibration procedure that focuses on the construction of minimal prediction sets according to a desired confidence level, to calibrate the parameter p as a function of the entropy of the next word distribution. We find that OPT models are overconfident, and that calibration shows a moderate inverse scaling with model size.

Key facts. Cited by 32 papers in the map (data through 13 Oct 2026). Builds on 14 of its 22 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

14 of 22 references.

References · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2020Scaling Laws for Neural Language Modelscited by 6,494
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,480
  4. 2017On Calibration of Modern Neural Networkscited by 4,902
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,500
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,893
  7. 2022Emergent Abilities of Large Language Modelscited by 2,499
  8. 2018Hierarchical Neural Story Generationcited by 1,186
  9. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 786
  10. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 468
  11. 2020Classification with Valid and Adaptive Coveragecited by 393
  12. 2022Truncation Sampling as Language Model Desmoothingcited by 121
2 more
  1. 2018Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Contextcited by 113
  2. 2022Inverse scaling can become U-shapedcited by 76

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 5; 2024: 13; 2025: 7; 2026: 7.

32 papers.

Sort:

By area:

  1. 2023Conformal Language Modelingcited by 167
  2. 2024Language Models with Conformal Factuality Guaranteescited by 131
  3. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 126
  4. 2024To Believe or Not to Believe Your LLMcited by 120
  5. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  6. 2024Mitigating LLM Hallucinations via Conformal Abstentioncited by 58
  7. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
  8. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curationcited by 44
  9. 2024Conformal Prediction: A Data Perspectivecited by 25
  10. 2024Prune 'n Predict: Optimizing LLM Decision-making with Conformal Predictioncited by 17
  11. 2023Conformal Autoregressive Generation: Beam Search with Coverage Guaranteescited by 16
  12. 2023Evaluating Machine Translation Quality with Conformal Predictive Distributionscited by 9
20 more
  1. 2024Priority Sampling of Large Language Models for Compilerscited by 8
  2. 2025CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimizationcited by 8
  3. 2023Conformalizing Machine Translation Evaluationcited by 5
  4. 2024Calibrated Large Language Models for Binary Question Answeringcited by 3
  5. 2024CONTESTS: a Framework for Consistency Testing of Span Probabilities in Language Modelscited by 3
  6. 2025Efficient semantic uncertainty quantification in language models via diversity-steered samplingcited by 2
  7. 2026Seed Hijacking of LLM Sampling and Quantum Random Number Defensecited by 2
  8. 2026AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoningcited by 2
  9. 2024Dynamic Skill Adaptation for Large Language Modelscited by 1
  10. 2025One Patient, Many Contexts: Scaling Medical AI with Contextual Intelligencecited by 1
  11. 2026Ailed: A Psyche-Driven Chess Engine with Dynamic Emotional Modulationcited by 1
  12. 2026Not All Errors Are Equal: A Systematic Study of Error Propagation in Large Language Model Inferencecited by 1
  13. 2024Synthesize, Partition, then Adapt: Eliciting Diverse Samples from Foundation Modelscited by 0
  14. 2025Leveraging LLMs for Predicting Unknown Diagnoses from Clinical Notescited by 0
  15. 2025Iterative Learning of Computable Phenotypes for Treatment Resistant Hypertension using Large Language Modelscited by 0
  16. 2025CoVeR: Conformal Calibration for Versatile and Reliable Autoregressive Next-Token Predictioncited by 0
  17. 2025The Semantic Illusion: Certified Limits of Embedding-Based Hallucination Detection in RAG Systemscited by 0
  18. 2026Entropy-Guided k-Guard Sampling for Long-Horizon Autoregressive Video Generationcited by 0
  19. 2026Full Conformal Prediction under Stochastic Non-Conformity Measurecited by 0
  20. 2026Conformity Breaks Conformal Predictioncited by 0