mlsanityData to 7 Oct 2026

Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity

arXiv 2602.00723 ↗First posted 31 Jan 2026cs.LG cs.AIcited by 1

On the map: LLM reasoning → Hallucination detection and mitigation

Abstract (arXiv)

Large language models (LLMs) are known to "hallucinate" by generating false or misleading outputs. Hallucinations pose various harms, from erosion of trust to widespread misinformation. Existing hallucination evaluation, however, focuses only on correctness and often overlooks consistency, necessary to distinguish and address these harms. To bridge this gap, we introduce prompt multiplicity, a framework for quantifying consistency in LLM evaluations. Our analysis reveals significant multiplicity (over 50% inconsistency in benchmarks like Med-HALT), suggesting that hallucination-related harms have been severely misunderstood. Furthermore, we study the role of consistency in hallucination detection and mitigation. We find that: (a) detection techniques detect consistency, not correctness, and (b) mitigation techniques like RAG, while beneficial, can introduce additional inconsistencies. By integrating prompt multiplicity into hallucination evaluation, we provide an improved framework of potential harms and uncover critical limitations in current detection and mitigation strategies.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 59 of its 75 references that are in the map. First posted on arXiv 31 Jan 2026.

Builds on

59 of 75 references.

References · 59

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  10. 2019Language Models as Knowledge Bases?cited by 1,922
  11. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  12. 2022Language Models (Mostly) Know What They Knowcited by 1,783
38 more
  1. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  4. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  5. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  6. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  7. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  8. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  9. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  10. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  11. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  12. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  13. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  14. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  15. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  16. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  17. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  18. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  19. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
  20. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  21. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 249
  22. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  23. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  24. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  25. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 179
  26. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  27. 2019Predictive Multiplicity in Classificationcited by 162
  28. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 156
  29. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 151
  30. 2024When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboardscited by 134
  31. 2024Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraphcited by 105
  32. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 103
  33. 2020Controlled Hallucinations: Learning to Generate Faithfully from Noisy Datacited by 83
  34. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 82
  35. 2024Efficient multi-prompt evaluation of LLMscited by 72
  36. 2024A Survey of Uncertainty Estimation in LLMs: Theory Meets Practicecited by 54
  37. 2023BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Modelscited by 53
  38. 2024Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden Statescited by 50

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CORTEX: Token-Level Hallucination Detection in RAG via Comparative Internal Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Theoretical Foundations and Mitigation of Hallucination in Large Language Models
  2. 2025Triggering Hallucinations in LLMs: A Quantitative Study of Prompt-Induced Hallucination in Large Language Models
  3. 2024THaMES: An End-to-End Tool for Hallucination Mitigation and Evaluation in Large Language Models
  4. 2026Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods
  5. 2024Fine-grained Hallucination Detection and Editing for Language Models
  6. 2026ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluation
  7. 2026Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgments
  8. 2025MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM