mlsanityData to 13 Oct 2026

A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation

arXiv 2307.03987 ↗First posted 8 Jul 2023cs.CLcited by 205

Not on the map. Why?

Abstract (arXiv)

Recently developed large language models have achieved remarkable success in generating fluent and coherent text. However, these models often tend to 'hallucinate' which critically hampers their reliability. In this work, we address this crucial problem and propose an approach that actively detects and mitigates hallucinations during the generation process. Specifically, we first identify the candidates of potential hallucination leveraging the model's logit output values, check their correctness through a validation procedure, mitigate the detected hallucinations, and then continue with the generation process. Through extensive experiments with GPT-3.5 (text-davinci-003) on the 'article generation task', we first demonstrate the individual efficacy of our detection and mitigation techniques. Specifically, the detection technique achieves a recall of ~88% and the mitigation technique successfully mitigates 57.6% of the correctly detected hallucinations. Importantly, our mitigation technique does not introduce new hallucinations even in the case of incorrectly detected hallucinations, i.e., false positives. Then, we show that the proposed active detection and mitigation approach successfully reduces the hallucinations of the GPT-3.5 model from 47.5% to 14.5% on average. We further demonstrate the effectiveness and wide applicability of our approach through additional studies including performance on different types of questions (multi-hop and false premise questions) and with another LLM from a different model family (Vicuna). In summary, our work contributes to improving the reliability and trustworthiness of large language models, a crucial step en route to enabling their widespread adoption in real-world applications.

Key facts. Cited by 205 papers in the map (data through 13 Oct 2026). Builds on 39 of its 51 references that are in the map. First posted on arXiv 8 Jul 2023.

Builds on

39 of 51 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,320
  2. 2022Training language models to follow instructions with human feedbackcited by 15,856
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,072
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,317
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,765
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,500
  9. 2019The Curious Case of Neural Text Degenerationcited by 2,893
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,068
  11. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,978
  12. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,758
27 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,404
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,201
  3. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 898
  4. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 865
  5. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 798
  6. 2021Deduplicating Training Data Makes Language Models Bettercited by 710
  7. 2023The Internal State of an LLM Knows When It's Lyingcited by 652
  8. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  9. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 556
  10. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  11. 2017Six Challenges for Neural Machine Translationcited by 335
  12. 2020Selective Question Answering under Domain Shiftcited by 231
  13. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  14. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 128
  15. 2019Handling Divergent Reference Texts when Evaluating Table-to-Text Generationcited by 119
  16. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  17. 2021UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmarkcited by 98
  18. 2021The Factual Inconsistency Problem in Abstractive Text Summarization: A Surveycited by 79
  19. 2021Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selectioncited by 76
  20. 2022Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settingscited by 66
  21. 2019Sticking to the Facts: Confident Decoding for Faithful Data-to-Text Generationcited by 62
  22. 2021Learning Rich Representation of Keyphrases from Textcited by 36
  23. 2021Focus Attention: Promoting Faithfulness and Diversity in Summarizationcited by 30
  24. 2020Revisiting Challenges in Data-to-Text Generation with Fact Groundingcited by 27
  25. 2023Interpretable Unified Language Checkingcited by 26
  26. 2023Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QAcited by 20
  27. 2019Modeling Fluency and Faithfulness for Diverse Neural Machine Translationcited by 13

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 29; 2024: 81; 2025: 62; 2026: 33.

205 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,594
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,433
  3. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 697
  4. 2023A Survey of Hallucination in Large Foundation Modelscited by 367
  5. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 320
  6. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 278
  7. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 268
  8. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 255
  9. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 255
  10. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 237
  11. 2024Capabilities of Gemini Models in Medicinecited by 236
  12. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 233
38 more
  1. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 205
  2. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  3. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 157
  4. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  5. 2024Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Modelscited by 114
  6. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 109
  7. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 107
  8. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 104
  9. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 102
  10. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100
  11. 2025Maximizing Confidence Alone Improves Reasoningcited by 99
  12. 2024Large language model validity via enhanced conformal prediction methodscited by 92
  13. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 90
  14. 2024The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Modelscited by 88
  15. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 86
  16. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
  17. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  18. 2024Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluationcited by 80
  19. 2024LLM-based NLG Evaluation: Current Status and Challengescited by 72
  20. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  21. 2023The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensivenesscited by 67
  22. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  23. 2024Factuality of Large Language Models: A Surveycited by 66
  24. 2025Reasoning Models Better Express Their Confidencecited by 65
  25. 2023Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Modelscited by 56
  26. 2023From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuningcited by 56
  27. 2025Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinationscited by 48
  28. 2024Multicalibration for Confidence Scoring in LLMscited by 45
  29. 2024Reducing Tool Hallucination via Reliability Alignmentcited by 39
  30. 2023Ever: Mitigating Hallucination in Large Language Models through Real-Time Verification and Rectificationcited by 38
  31. 2024Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinationscited by 38
  32. 2024LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraintscited by 35
  33. 2023Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledgecited by 32
  34. 2024Teaching Large Language Models to Express Knowledge Boundary from Their Own Signalscited by 32
  35. 2024Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questionscited by 30
  36. 2024Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuningcited by 30
  37. 2024Evolutionary Large Language Model for Automated Feature Transformationcited by 29
  38. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28

50 most cited of 205 All 205 (JSON)