mlsanityData to 7 Oct 2026

Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation

arXiv 2305.15852 ↗First posted 25 May 2023cs.CL cs.AI cs.LGcited by 123

Not on the map. Why?

Cited by 26 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (large LMs) are susceptible to producing text that contains hallucinated content. An important instance of this problem is self-contradiction, where the LM generates two contradictory sentences within the same context. In this work, we present a comprehensive investigation into self-contradiction for various instruction-tuned LMs, covering evaluation, detection, and mitigation. Our primary evaluation task is open-domain text generation, but we also demonstrate the applicability of our approach to shorter question answering. Our analysis reveals the prevalence of self-contradictions, e.g., in 17.7% of all sentences produced by ChatGPT. We then propose a novel prompting-based framework designed to effectively detect and mitigate self-contradictions. Our detector achieves high accuracy, e.g., around 80% F1 score when prompting ChatGPT. The mitigation algorithm iteratively refines the generated text to remove contradictory information while preserving text fluency and informativeness. Importantly, our entire framework is applicable to black-box LMs and does not require retrieval of external knowledge. Rather, our method complements retrieval-based methods, as a large portion of self-contradictions (e.g., 35.2% for ChatGPT) cannot be verified using online text. Our approach is practically effective and has been released as a push-button tool to benefit the public at https://chatprotect.ai/.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 53 of its 80 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

53 of 80 references.

References · 53

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  8. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
38 more
  1. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  4. 2019Language Models as Knowledge Bases?cited by 1,922
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  6. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  7. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  8. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  9. 2021Ethical and social risks of harm from Language Modelscited by 957
  10. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  11. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  12. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  13. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  14. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  15. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  16. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  17. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  18. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  19. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  20. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  21. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  22. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  23. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4cited by 202
  24. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  25. 2023A Categorical Archive of ChatGPT Failurescited by 167
  26. 2021Hurdles to Progress in Long-form Question Answeringcited by 166
  27. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  28. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 155
  29. 2020Detecting Hallucinated Content in Conditional Neural Sequence Generationcited by 151
  30. 2018Dialogue Natural Language Inferencecited by 135
  31. 2020Factual Error Correction for Abstractive Summarization Modelscited by 108
  32. 2023Crawling the Internal Knowledge-Base of Language Modelscited by 85
  33. 2021The Factual Inconsistency Problem in Abstractive Text Summarization: A Surveycited by 79
  34. 2019Evaluating Coherence in Dialogue Systems using Entailmentcited by 53
  35. 2022Read before Generate! Faithful Long Form Question Answering with Machine Readingcited by 47
  36. 2023MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarizationcited by 30
  37. 2022LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Modelscited by 28
  38. 2021Think Before You Speak: Explicitly Generating Implicit Commonsense Knowledge for Response Generationcited by 26

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 26; 2024: 52; 2025: 28; 2026: 17.

123 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  3. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  4. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  5. 2023Fine-tuning Language Models for Factualitycited by 219
  6. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  7. 2023Can LLM-Generated Misinformation Be Detected?cited by 173
  8. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  9. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  10. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  11. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 98
  12. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 96
38 more
  1. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 86
  2. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 85
  3. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 85
  4. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  5. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 82
  6. 2023Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Modelscited by 67
  7. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 64
  8. 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoningcited by 59
  9. 2024InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignmentcited by 56
  10. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  11. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
  12. 2024AlphaFin: Benchmarking Financial Analysis with Retrieval-Augmented Stock-Chain Frameworkcited by 47
  13. 2023Faithful Persona-based Conversational Dataset Generation with Large Language Modelscited by 45
  14. 2023Chainpoll: A high efficacy method for LLM hallucination detectioncited by 39
  15. 2024Reducing Tool Hallucination via Reliability Alignmentcited by 37
  16. 2023Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Modelscited by 36
  17. 2023Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinationscited by 34
  18. 2024Measuring and Reducing LLM Hallucination without Gold-Standard Answerscited by 31
  19. 2024GraphEval: A Knowledge-Graph Based LLM Hallucination Evaluation Frameworkcited by 30
  20. 2024Improving Uncertainty Quantification in Large Language Models via Semantic Embeddingscited by 30
  21. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
  22. 2024I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Tokencited by 26
  23. 2024COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Actcited by 23
  24. 2024LLMDFA: Analyzing Dataflow in Code with Large Language Modelscited by 22
  25. 2024Formal Theorem Proving by Rewarding LLMs to Decompose Proofs Hierarchicallycited by 22
  26. 2024A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagationcited by 21
  27. 2024Atomic Fact Decomposition Helps Attributed Question Answeringcited by 21
  28. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Modelscited by 21
  29. 2024Hallucination Detection and Hallucination Mitigation: An Investigationcited by 20
  30. 2024Estimating the Hallucination Rate of Generative AIcited by 20
  31. 2023Zero-Resource Hallucination Prevention for Large Language Modelscited by 19
  32. 2025Robust Hallucination Detection in LLMs via Adaptive Token Selectioncited by 19
  33. 2025ToolFuzz -- Automated Agent Tool Testingcited by 18
  34. 2025Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Modelscited by 18
  35. 2024RadFlag: A Black-Box Hallucination Detection Method for Medical Vision Language Modelscited by 16
  36. 2023Self-Contradictory Reasoning Evaluation and Detectioncited by 14
  37. 2024Large Language Model for Mental Health: A Systematic Reviewcited by 14
  38. 2024PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamicscited by 14

50 most cited of 123 All 123 (JSON)