# Self-Evaluation Improves Selective Generation in Large Language Models

arXiv: https://arxiv.org/abs/2312.09300 · HTML: https://mlsanity.com/paper/2312.09300/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 34 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [Scaling Out-of-Distribution Detection for Real-World Settings](https://mlsanity.com/paper/1911.11132/) (2019)
- [SLiC-HF: Sequence Likelihood Calibration with Human Feedback](https://mlsanity.com/paper/2305.10425/) (2023)
- [Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners](https://mlsanity.com/paper/2307.01928/) (2023)
- [Leveraging Large Language Models for Multiple Choice Question Answering](https://mlsanity.com/paper/2210.12353/) (2022)
- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [Out-of-Distribution Detection and Selective Generation for Conditional Language Models](https://mlsanity.com/paper/2209.15558/) (2022)
- [BRIO: Bringing Order to Abstractive Summarization](https://mlsanity.com/paper/2203.16804/) (2022)
- [Selectively Answering Ambiguous Questions](https://mlsanity.com/paper/2305.14613/) (2023)
- [On Uncertainty Calibration and Selective Generation in Probabilistic Neural Summarization: A Benchmark Study](https://mlsanity.com/paper/2304.08653/) (2023)
- [Collaborative Storytelling with Large-scale Neural Language Models](https://mlsanity.com/paper/2011.10208/) (2020)
- [Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimation](https://mlsanity.com/paper/2107.04212/) (2021)
- [Can NLP Models 'Identify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer?](https://mlsanity.com/paper/2309.04635/) (2023)

## Built on by (most cited first, 30 of 76)

- [Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning](https://mlsanity.com/paper/2405.00451/) (2024)
- [Deep Think with Confidence](https://mlsanity.com/paper/2508.15260/) (2025)
- [Scalable Best-of-N Selection for Large Language Models via Self-Certainty](https://mlsanity.com/paper/2502.18581/) (2025)
- [Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities](https://mlsanity.com/paper/2405.20003/) (2024)
- [LLM Post-Training: A Deep Dive into Reasoning Large Language Models](https://mlsanity.com/paper/2502.21321/) (2025)
- [Know Your Limits: A Survey of Abstention in Large Language Models](https://mlsanity.com/paper/2407.18418/) (2024)
- [HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection](https://mlsanity.com/paper/2409.17504/) (2024)
- [Interpretable Contrastive Monte Carlo Tree Search Reasoning](https://mlsanity.com/paper/2410.01707/) (2024)
- [SETS: Leveraging Self-Verification and Self-Correction for Improved Test-Time Scaling](https://mlsanity.com/paper/2501.19306/) (2025)
- [Improving Uncertainty Estimation through Semantically Diverse Language Generation](https://mlsanity.com/paper/2406.04306/) (2024)
- [Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics](https://mlsanity.com/paper/2502.15815/) (2025)
- [Does It Make Sense to Speak of Introspection in Large Language Models?](https://mlsanity.com/paper/2506.05068/) (2025)
- [DataGen: Unified Synthetic Dataset Generation via Large Language Models](https://mlsanity.com/paper/2406.18966/) (2024)
- [A Survey on the Honesty of Large Language Models](https://mlsanity.com/paper/2409.18786/) (2024)
- [Multi-Agent Debate for LLM Judges with Adaptive Stability Detection](https://mlsanity.com/paper/2510.12697/) (2025)
- [LOVA3: Learning to Visual Question Answering, Asking and Assessment](https://mlsanity.com/paper/2405.14974/) (2024)
- [Maestro: Self-Improving Text-to-Image Generation via Agent Orchestration](https://mlsanity.com/paper/2509.10704/) (2025)
- [Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in Its Latent Thoughts](https://mlsanity.com/paper/2509.26314/) (2025)
- [Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measure](https://mlsanity.com/paper/2412.15176/) (2024)
- [Uncertainty Quantification for Language Models: A Suite of Black-Box, White-Box, LLM Judge, and Ensemble Scorers](https://mlsanity.com/paper/2504.19254/) (2025)
- [On Subjective Uncertainty Quantification and Calibration in Natural Language Generation](https://mlsanity.com/paper/2406.05213/) (2024)
- [Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners](https://mlsanity.com/paper/2406.00430/) (2024)
- [Agentic Keyframe Search for Video Question Answering](https://mlsanity.com/paper/2503.16032/) (2025)
- [Matchmaker: Self-Improving Large Language Model Programs for Schema Matching](https://mlsanity.com/paper/2410.24105/) (2024)
- [BioAgents: Democratizing Bioinformatics Analysis with Multi-Agent Systems](https://mlsanity.com/paper/2501.06314/) (2025)
- [HALT: Hallucination Assessment via Log-probs as Time series](https://mlsanity.com/paper/2602.02888/) (2026)
- [Evaluation of LLMs for mathematical problem solving](https://mlsanity.com/paper/2506.00309/) (2025)
- [LLMs can learn self-restraint through iterative self-reflection](https://mlsanity.com/paper/2405.13022/) (2024)
- [GenCLS++: Pushing the Boundaries of Generative Classification in LLMs Through Comprehensive SFT and RL Studies Across Diverse Datasets](https://mlsanity.com/paper/2504.19898/) (2025)
- [Autonomous Prompt Engineering in Large Language Models](https://mlsanity.com/paper/2407.11000/) (2024)

## Abstract (arXiv)

Safe deployment of large language models (LLMs) may benefit from a reliable method for assessing their generated content to determine when to abstain or to selectively generate. While likelihood-based metrics such as perplexity are widely employed, recent research has demonstrated the limitations of using sequence-level probability estimates given by LLMs as reliable indicators of generation quality. Conversely, LLMs have demonstrated strong calibration at the token level, particularly when it comes to choosing correct answers in multiple-choice questions or evaluating true/false statements. In this work, we reformulate open-ended generation tasks into token-level prediction tasks, and leverage LLMs' superior calibration at the token level. We instruct an LLM to self-evaluate its answers, employing either a multi-way comparison or a point-wise evaluation approach, with the option to include a ``None of the above'' option to express the model's uncertainty explicitly. We benchmark a range of scoring methods based on self-evaluation and evaluate their performance in selective generation using TruthfulQA and TL;DR. Through experiments with PaLM-2 and GPT-3, we demonstrate that self-evaluation based scores not only improve accuracy, but also correlate better with the overall quality of generated content.