mlsanityData to 14 Oct 2026

The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models

arXiv 2404.05904 ↗First posted 8 Apr 2024cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have transformed the Natural Language Processing (NLP) landscape with their remarkable ability to understand and generate human-like text. However, these models are prone to ``hallucinations'' -- outputs that do not align with factual reality or the input context. This paper introduces the Hallucinations Leaderboard, an open initiative to quantitatively measure and compare the tendency of each model to produce hallucinations. The leaderboard uses a comprehensive set of benchmarks focusing on different aspects of hallucinations, such as factuality and faithfulness, across various tasks, including question-answering, summarisation, and reading comprehension. Our analysis provides insights into the performance of different models, guiding researchers and practitioners in choosing the most reliable models for their applications.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 8 Apr 2024.

Builds on

41 of 58 references.

References · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  10. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  11. 2019Language Models as Knowledge Bases?cited by 1,936
  12. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
29 more
  1. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  2. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  5. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  6. 2019How Can We Know What Language Models Know?cited by 959
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  8. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 861
  9. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  10. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  11. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  12. 2023The Internal State of an LLM Knows When It's Lyingcited by 659
  13. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 584
  14. 2023The Falcon Series of Open Language Modelscited by 535
  15. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
  16. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
  17. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  18. 2023How Language Model Hallucinations Can Snowballcited by 311
  19. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 310
  20. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 244
  21. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 156
  22. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  23. 2023SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistencycited by 90
  24. 2024Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvementscited by 83
  25. 2023FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledgecited by 55
  26. 2022Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusterscited by 44
  27. 2022FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representationscited by 41
  28. 2022Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarizationcited by 34
  29. 2021Relational World Knowledge Representation in Contextual Language Models: A Reviewcited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 2.

17 papers.

Sort:

By area:

  1. 2024Logically Consistent Language Models via Neuro-Symbolic Integrationcited by 30
  2. 2024DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinationscited by 25
  3. 2025Conformal Linguistic Calibration: Trading-off between Factuality and Specificitycited by 20
  4. 2025Malicious LLM-Based Conversational AI Makes Users Reveal Personal Informationcited by 13
  5. 2025MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systemscited by 11
  6. 2024A Survey of Automatic Hallucination Evaluation on Natural Language Generationcited by 7
  7. 2025Self-Training Large Language Models for Tool-Use Without Demonstrationscited by 7
  8. 2025XtraGPT: Context-Aware and Controllable Academic Paper Revision via Human-AI Collaborationcited by 7
  9. 2025Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Promptscited by 5
  10. 2025SelfCheck-Eval: A Multi-Module Framework for Zero-Resource Hallucination Detection in Large Language Modelscited by 4
  11. 2024ViBe: A Text-to-Video Benchmark for Evaluating Hallucination in Large Multimodal Modelscited by 3
  12. 2025An Analysis of Decoding Methods for LLM-based Agents for Faithful Multi-Hop Question Answeringcited by 2
5 more
  1. 2025Neural Diversity Regularizes Hallucinations in Language Modelscited by 2
  2. 2026Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicitycited by 2
  3. 2025Simulated Self-Assessment in Large Language Models: A Psychometric Approach to AI Self-Efficacycited by 1
  4. 2024MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systemscited by 0
  5. 2026Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New Insightscited by 0