# Are Large Language Models Good Fact Checkers: A Preliminary Study

arXiv: https://arxiv.org/abs/2311.17355 · HTML: https://mlsanity.com/paper/2311.17355/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization](https://mlsanity.com/paper/2309.07430/) (2023)
- [Fact-Checking Complex Claims with Program-Guided Reasoning](https://mlsanity.com/paper/2305.12744/) (2023)
- [Automated Fact Checking: Task formulations, methods and future directions](https://mlsanity.com/paper/1806.07687/) (2018)
- [Towards Few-Shot Fact-Checking via Perplexity](https://mlsanity.com/paper/2103.09535/) (2021)
- [FactKG: Fact Verification via Reasoning on Knowledge Graphs](https://mlsanity.com/paper/2305.06590/) (2023)
- [Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2109.11427/) (2021)
- [GERE: Generative Evidence Retrieval for Fact Verification](https://mlsanity.com/paper/2204.05511/) (2022)
- [CHEF: A Pilot Chinese Dataset for Evidence-Based Fact-Checking](https://mlsanity.com/paper/2206.11863/) (2022)
- [Check-COVID: Fact-Checking COVID-19 News Claims with Scientific Evidence](https://mlsanity.com/paper/2305.18265/) (2023)
- [Fin-Fact: A Benchmark Dataset for Multimodal Financial Fact Checking and Explanation Generation](https://mlsanity.com/paper/2309.08793/) (2023)
- [Read it Twice: Towards Faithfully Interpretable Fact Verification by Revisiting Evidence](https://mlsanity.com/paper/2305.03507/) (2023)
- [Explainable Fact-checking through Question Answering](https://mlsanity.com/paper/2110.05369/) (2021)
- [Exploring Faithful Rationale for Multi-hop Fact Verification via Salience-Aware Graph Learning](https://mlsanity.com/paper/2212.01060/) (2022)
- [From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verification](https://mlsanity.com/paper/2310.11675/) (2023)
- [Multi-hop Evidence Retrieval for Cross-document Relation Extraction](https://mlsanity.com/paper/2212.10786/) (2022)
- [Leveraging LLMs in Scholarly Knowledge Graph Question Answering](https://mlsanity.com/paper/2311.09841/) (2023)
- [Claim-Dissector: An Interpretable Fact-Checking System with Joint Re-ranking and Veracity Prediction](https://mlsanity.com/paper/2207.14116/) (2022)
- [Natural Logic-guided Autoregressive Multi-hop Document Retrieval for Fact Verification](https://mlsanity.com/paper/2212.05276/) (2022)
- [Table-based Fact Verification with Salience-aware Learning](https://mlsanity.com/paper/2109.04053/) (2021)
- [Logic-level Evidence Retrieval and Graph-based Verification Network for Table-based Fact Verification](https://mlsanity.com/paper/2109.06480/) (2021)
- [AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learning](https://mlsanity.com/paper/2311.12371/) (2023)

## Built on by (most cited first, 8 of 8)

- [DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts](https://mlsanity.com/paper/2412.10510/) (2024)
- [Generative Large Language Models in Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2407.02351/) (2024)
- [ERBench: An Entity-Relationship based Automatically Verifiable Hallucination Benchmark for Large Language Models](https://mlsanity.com/paper/2403.05266/) (2024)
- [CMIE: Combining MLLM Insights with External Evidence for Explainable Out-of-Context Misinformation Detection](https://mlsanity.com/paper/2505.23449/) (2025)
- [How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models](https://mlsanity.com/paper/2407.00369/) (2024)
- [When Scale Meets Diversity: Evaluating Language Models on Fine-Grained Multilingual Claim Verification](https://mlsanity.com/paper/2507.20700/) (2025)
- [ZeFaV: Boosting Large Language Models for Zero-shot Fact Verification](https://mlsanity.com/paper/2411.11247/) (2024)
- [Can LLMs Take Retrieved Information with a Grain of Salt?](https://mlsanity.com/paper/2605.06919/) (2026)

## Abstract (arXiv)

Recently, Large Language Models (LLMs) have drawn significant attention due to their outstanding reasoning capabilities and extensive knowledge repository, positioning them as superior in handling various natural language processing tasks compared to other language models. In this paper, we present a preliminary investigation into the potential of LLMs in fact-checking. This study aims to comprehensively evaluate various LLMs in tackling specific fact-checking subtasks, systematically evaluating their capabilities, and conducting a comparative analysis of their performance against pre-trained and state-of-the-art low-parameter models. Experiments demonstrate that LLMs achieve competitive performance compared to other small models in most scenarios. However, they encounter challenges in effectively handling Chinese fact verification and the entirety of the fact-checking pipeline due to language inconsistencies and hallucinations. These findings underscore the need for further exploration and research to enhance the proficiency of LLMs as reliable fact-checkers, unveiling the potential capability of LLMs and the possible challenges in fact-checking tasks.