mlsanityData to 14 Oct 2026

Sources of Hallucination by Large Language Models on Inference Tasks

arXiv 2305.14552 ↗First posted 23 May 2023cs.CL cs.AIcited by 169

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are claimed to be capable of Natural Language Inference (NLI), necessary for applied tasks like question answering and summarization. We present a series of behavioral studies on several LLM families (LLaMA, GPT-3.5, and PaLM) which probe their behavior using controlled experiments. We establish two biases originating from pretraining which predict much of their behavior, and show that these are major sources of hallucination in generative LLMs. First, memorization at the level of sentences: we show that, regardless of the premise, models falsely label NLI test samples as entailing when the hypothesis is attested in training data, and that entities are used as ``indices'' to access the memorized data. Second, statistical patterns of usage learned at the level of corpora: we further show a similar effect when the premise predicate is less frequent than that of the hypothesis in the training data, a bias following from previous studies. We demonstrate that LLMs perform significantly worse on NLI test samples which do not conform to these biases than those which do, and we offer these as valuable controls for future LLM evaluation.

Key facts. Cited by 169 papers in the map (data through 14 Oct 2026). Builds on 24 of its 46 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

24 of 46 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  9. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  11. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  12. 2023Benchmarking Large Language Models for News Summarizationcited by 445
12 more
  1. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 326
  2. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 295
  3. 2022Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Modelscited by 220
  4. 2022Language models show human-like content effects on reasoning taskscited by 173
  5. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
  6. 2022QUILL: Query Intent with Large Language Models using Retrieval Augmentation and Multi-stage Distillationcited by 21
  7. 2019Probabilistic Models of Relational Implicationcited by 17
  8. 2018Testing the Generalization Power of Neural Network Models Across NLI Benchmarkscited by 15
  9. 2021Multivalent Entailment Graphs for Question Answeringcited by 10
  10. 2021Language Models for Lexical Inference in Contextcited by 9
  11. 2022Smoothing Entailment Graphs with Language Modelscited by 8
  12. 2022Language Models Are Poor Learners of Directional Inferencecited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 48; 2024: 50; 2025: 53; 2026: 18.

169 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  3. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  4. 2024Large Language Models: A Surveycited by 525
  5. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  6. 2023Explainability for Large Language Models: A Surveycited by 333
  7. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  8. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 225
  9. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 189
  10. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 183
  11. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  12. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 163
38 more
  1. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 142
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  3. 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
  4. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 105
  5. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100
  6. 2024Bias and Unfairness in Information Retrieval Systems: New Challenges in the LLM Eracited by 98
  7. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 90
  8. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 84
  9. 2023Towards Mitigating Hallucination in Large Language Models via Self-Reflectioncited by 81
  10. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 79
  11. 2023On the Calibration of Large Language Models and Alignmentcited by 73
  12. 2025Process vs. Outcome Reward: Which is Better for Agentic RAG Reinforcement Learningcited by 70
  13. 2023Large Language Models can Learn Rulescited by 69
  14. 2023Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4cited by 68
  15. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  16. 2023Large Language Models for Supply Chain Optimizationcited by 64
  17. 2023HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Modelscited by 63
  18. 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoningcited by 59
  19. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 56
  20. 2024Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigationcited by 44
  21. 2023A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detectioncited by 42
  22. 2023Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinationscited by 35
  23. 2023How are Prompts Different in Terms of Sensitivity?cited by 35
  24. 2023Trapping LLM Hallucinations Using Tagged Context Promptscited by 34
  25. 2023CoQuest: Exploring Research Question Co-Creation with an LLM-based Agentcited by 34
  26. 2023Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AIcited by 32
  27. 2023Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledgecited by 32
  28. 2024AI-LieDar: Examine the Trade-off Between Utility and Truthfulness in LLM Agentscited by 31
  29. 2023Improving the Reliability of Large Language Models by Leveraging Uncertainty-Aware In-Context Learningcited by 29
  30. 2024Humanizing Machine-Generated Content: Evading AI-Text Detection through Adversarial Attackcited by 27
  31. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
  32. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 26
  33. 2023Probing the Moral Development of Large Language Models through Defining Issues Testcited by 25
  34. 2025Sparks of Science: Hypothesis Generation Using Structured Paper Datacited by 25
  35. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Datacited by 24
  36. 2023MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansioncited by 23
  37. 2024Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Taskscited by 23
  38. 2024Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)cited by 20

50 most cited of 169 All 169 (JSON)