mlsanityData to 7 Oct 2026

Are Large Language Models Good Fact Checkers: A Preliminary Study

arXiv 2311.17355 ↗First posted 29 Nov 2023cs.CLcited by 8

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Recently, Large Language Models (LLMs) have drawn significant attention due to their outstanding reasoning capabilities and extensive knowledge repository, positioning them as superior in handling various natural language processing tasks compared to other language models. In this paper, we present a preliminary investigation into the potential of LLMs in fact-checking. This study aims to comprehensively evaluate various LLMs in tackling specific fact-checking subtasks, systematically evaluating their capabilities, and conducting a comparative analysis of their performance against pre-trained and state-of-the-art low-parameter models. Experiments demonstrate that LLMs achieve competitive performance compared to other small models in most scenarios. However, they encounter challenges in effectively handling Chinese fact verification and the entirety of the fact-checking pipeline due to language inconsistencies and hallucinations. These findings underscore the need for further exploration and research to enhance the proficiency of LLMs as reliable fact-checkers, unveiling the potential capability of LLMs and the possible challenges in fact-checking tasks.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 29 of its 43 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

29 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Fact-Checking Complex Claims with Program-Guided Reasoning22 May

Other references · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  7. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  8. 2021A Survey on Automated Fact-Checkingcited by 376
  9. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 244
  10. 2018Automated Fact Checking: Task formulations, methods and future directionscited by 96
  11. 2021Towards Few-Shot Fact-Checking via Perplexitycited by 63
  12. 2023FactKG: Fact Verification via Reasoning on Knowledge Graphscited by 57
16 more
  1. 2021Automated Fact-Checking: A Surveycited by 55
  2. 2022GERE: Generative Evidence Retrieval for Fact Verificationcited by 38
  3. 2022CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checkingcited by 37
  4. 2023Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidencecited by 25
  5. 2023Fin-Fact: A Benchmark Dataset for Multimodal Financial Fact Checking and Explanation Generationcited by 25
  6. 2023Read it Twice: Towards Faithfully Interpretable Fact Verification by Revisiting Evidencecited by 17
  7. 2021Explainable Fact-checking through Question Answeringcited by 16
  8. 2022Exploring Faithful Rationale for Multi-hop Fact Verification via Salience-Aware Graph Learningcited by 15
  9. 2023From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verificationcited by 15
  10. 2022Multi-hop Evidence Retrieval for Cross-document Relation Extractioncited by 13
  11. 2023Leveraging LLMs in Scholarly Knowledge Graph Question Answeringcited by 13
  12. 2022Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Predictioncited by 12
  13. 2022Natural Logic-guided Autoregressive Multi-hop Document Retrieval for Fact Verificationcited by 11
  14. 2021Table-based Fact Verification with Salience-aware Learningcited by 9
  15. 2021Logic-level Evidence Retrieval and Graph-based Verification Network for Table-based Fact Verificationcited by 4
  16. 2023AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 2; 2026: 1.

8 papers.

  1. 2024DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Expertscited by 42
  2. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19
  3. 2024ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Modelscited by 9
  4. 2025CMIE: Combining MLLM Insights with External Evidence for Explainable Out-of-Context Misinformation Detectioncited by 6
  5. 2024How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Modelscited by 3
  6. 2025When Scale Meets Diversity: Evaluating Language Models on Fine-Grained Multilingual Claim Verificationcited by 3
  7. 2024ZeFaV: Boosting Large Language Models for Zero-shot Fact Verificationcited by 1
  8. 2026Can LLMs Take Retrieved Information with a Grain of Salt?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CANDY: Benchmarking LLMs' Limitations and Assistive Potential in Chinese Misinformation Fact-Checking
  2. 2024Evaluating Large Language Model Capability in Vietnamese Fact-Checking Data Generation
  3. 2023The Perils & Promises of Fact-checking with Large Language Models
  4. 2024Logical Consistency of Large Language Models in Fact-checking
  5. 2025RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checking
  6. 2026Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checking
  7. 2026InFi-Check: Interpretable and Fine-Grained Fact-Checking of LLMs
  8. 2024Chinese SafetyQA: A Safety Short-form Factuality Benchmark for Large Language Models