mlsanityData to 14 Oct 2026

Do Large Language Models Know about Facts?

arXiv 2310.05177 ↗First posted 8 Oct 2023cs.CLcited by 51

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have recently driven striking performance improvements across a range of natural language processing tasks. The factual knowledge acquired during pretraining and instruction tuning can be useful in various downstream tasks, such as question answering, and language generation. Unlike conventional Knowledge Bases (KBs) that explicitly store factual knowledge, LLMs implicitly store facts in their parameters. Content generated by the LLMs can often exhibit inaccuracies or deviations from the truth, due to facts that can be incorrectly induced or become obsolete over time. To this end, we aim to comprehensively evaluate the extent and scope of factual knowledge within LLMs by designing the benchmark Pinocchio. Pinocchio contains 20K diverse factual questions that span different sources, timelines, domains, regions, and languages. Furthermore, we investigate whether LLMs are able to compose multiple facts, update factual knowledge temporally, reason over multiple pieces of facts, identify subtle factual differences, and resist adversarial examples. Extensive experiments on different sizes and types of LLMs show that existing LLMs still lack factual knowledge and suffer from various spurious correlations. We believe this is a critical bottleneck for realizing trustworthy artificial intelligence. The dataset Pinocchio and our codes will be publicly available.

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Builds on 56 of its 65 references that are in the map. First posted on arXiv 8 Oct 2023.

Builds on

56 of 65 references.

References · 56

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  12. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
38 more
  1. 2021Program Synthesis with Large Language Modelscited by 3,637
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  8. 2019Language Models as Knowledge Bases?cited by 1,936
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  10. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  11. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  12. 2019How Can We Know What Language Models Know?cited by 959
  13. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  14. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  15. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  16. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  17. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  18. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  19. 2023In-Context Retrieval-Augmented Language Modelscited by 702
  20. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 488
  21. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  22. 2021A Survey on Automated Fact-Checkingcited by 378
  23. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  24. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 298
  25. 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
  26. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 225
  27. 2016Model-Agnostic Interpretability of Machine Learningcited by 219
  28. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 211
  29. 2020How Context Affects Language Models' Factual Predictionscited by 181
  30. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 167
  31. 2023Fact-Checking Complex Claims with Program-Guided Reasoningcited by 146
  32. 2019Towards Debiasing Fact Verification Modelscited by 143
  33. 2023On the Tool Manipulation Capability of Open-source Large Language Modelscited by 122
  34. 2020X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Modelscited by 117
  35. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  36. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  37. 2022SciFact-Open: Towards open-domain scientific claim verificationcited by 94
  38. 2022Calibrating Factual Knowledge in Pretrained Language Modelscited by 92

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 27; 2025: 12; 2026: 7.

51 papers.

Sort:

By area:

  1. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  2. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  3. 2023A Semantic Invariant Robust Watermark for Large Language Modelscited by 112
  4. 2024Large Language Models Can Learn Temporal Reasoningcited by 111
  5. 2024KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graphcited by 93
  6. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 88
  7. 2023Alleviating Hallucinations of Large Language Models through Induced Hallucinationscited by 79
  8. 2024Unified Hallucination Detection for Multimodal Large Language Modelscited by 75
  9. 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemscited by 72
  10. 2024CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Modelscited by 54
  11. 2024ChartInsights: Evaluating Multimodal Large Language Models for Low-Level Chart Question Answeringcited by 50
  12. 2024LLMArena: Assessing Capabilities of Large Language Models in Dynamic Multi-Agent Environmentscited by 36
38 more
  1. 2024Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Modelscited by 30
  2. 2023FactCHD: Benchmarking Fact-Conflicting Hallucination Detectioncited by 29
  3. 2023Are Large Language Models Temporally Grounded?cited by 27
  4. 2024Citation-Enhanced Generation for LLM-based Chatbotscited by 25
  5. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19
  6. 2024Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparatorscited by 19
  7. 2025FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Modelscited by 19
  8. 2024Do We Need Language-Specific Fact-Checking Models? The Case of Chinesecited by 15
  9. 2024MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Modelscited by 15
  10. 2024Multimodal Large Language Models to Support Real-World Fact-Checkingcited by 13
  11. 2024Towards Faithful and Robust LLM Specialists for Evidence-Based Question-Answeringcited by 11
  12. 2024Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understandingcited by 11
  13. 2024Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphscited by 10
  14. 2024Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verificationcited by 9
  15. 2024Deep Structural Knowledge Exploitation and Synergy for Estimating Node Importance Value on Heterogeneous Information Networkscited by 9
  16. 2024ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Modelscited by 9
  17. 2024Evaluating Robustness of Generative Search Engine on Adversarial Factual Questionscited by 9
  18. 2025RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checkingcited by 9
  19. 2025LLMs Deceive Unintentionally: Emergent Misalignment in Dishonesty from Misaligned Samples to Biased Human-AI Interactionscited by 8
  20. 2025Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Datacited by 5
  21. 2024LTLBench: Towards Benchmarks for Evaluating Temporal Reasoning in Large Language Modelscited by 4
  22. 2025Beyond Memorization: A Rigorous Evaluation Framework for Medical Knowledge Editingcited by 4
  23. 2024Navigating the Noisy Crowd: Finding Key Information for Claim Verificationcited by 3
  24. 2024Gradient Localization Improves Lifelong Pretraining of Language Modelscited by 3
  25. 2025When Scale Meets Diversity: Evaluating Language Models on Fine-Grained Multilingual Claim Verificationcited by 3
  26. 2026Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checkingcited by 3
  27. 2026CoG: Controllable Graph Reasoning via Relational Blueprints and Failure-Aware Refinement over Knowledge Graphscited by 3
  28. 2025Can Performant LLMs Be Ethical? Quantifying the Impact of Web Crawling Opt-Outscited by 2
  29. 2025Conflicts in Texts: Data, Implications and Challengescited by 2
  30. 2025From Confidence to Collapse in LLM Factual Robustnesscited by 2
  31. 2024A Comparative Study of Translation Bias and Accuracy in Multilingual Large Language Models for Cross-Language Claim Verificationcited by 1
  32. 2024Are LLMs Really Not Knowledgeable? Mining the Submerged Knowledge in LLMs' Memorycited by 1
  33. 2025Efficient Dynamic Clustering-Based Document Compression for Retrieval-Augmented-Generationcited by 1
  34. 2025HA-RAG: Hotness-Aware RAG Acceleration via Mixed Precision and Data Placementcited by 0
  35. 2026DHI: Leveraging Diverse Hallucination Induction for Enhanced Contrastive Factuality Control in Large Language Modelscited by 0
  36. 2026Tug-of-War within A Decade: Conflict Resolution in Vulnerability Analysis via Teacher-Guided Retrieval-Augmented Generationscited by 0
  37. 2026Detecting and Explaining Fake News Short Videos with Multimodal Content and Real-World Evidencecited by 0
  38. 2026Large-scale factor analysis shows machine intelligence is only partially interpretablecited by 0

50 most cited of 51 All 51 (JSON)