mlsanityData to 7 Oct 2026

How LLMs Fail to Support Fact-Checking

arXiv 2503.01902 ↗First posted 28 Feb 2025cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

While Large Language Models (LLMs) can amplify online misinformation, they also show promise in tackling misinformation. In this paper, we empirically study the capabilities of three LLMs -- ChatGPT, Gemini, and Claude -- in countering political misinformation. We implement a two-step, chain-of-thought prompting approach, where models first identify credible sources for a given claim and then generate persuasive responses. Our findings suggest that models struggle to ground their responses in real news sources, and tend to prefer citing left-leaning sources. We also observe varying degrees of response diversity among models. Our findings highlight concerns about using LLMs for fact-checking through only prompt-engineering, emphasizing the need for more robust guardrails. Our results have implications for both researchers and non-technical users.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 9 of its 18 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

9 of 18 references.

References · 9

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  4. 2023Whose Opinions Do Language Models Reflect?cited by 759
  5. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  6. 2023Factuality Challenges in the Era of Large Language Modelscited by 125
  7. 2024Large Language Models Produce Responses Perceived to be Empathiccited by 36
  8. 2023Fact-checking information from large language models can decrease headline discernmentcited by 22
  9. 2023FakeGPT: Fake News Generation, Explanation and Detection of Large Language Modelscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026Replicating Human Motivated Reasoning Studies with LLMscited by 5
  2. 2025Multi-Agent Retrieval-Augmented Framework for Evidence-Based Counterspeech Against Health Misinformationcited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2025Large Language Models Require Curated Context for Reliable Political Fact-Checking -- Even with Reasoning and Web Search
  2. 2024From Deception to Detection: The Dual Roles of Large Language Models in Fake News
  3. 2025Fact-checking AI-generated news reports: Can LLMs catch their own lies?
  4. 2023The Perils & Promises of Fact-checking with Large Language Models
  5. 2025Understanding Inequality of LLM Fact-Checking over Geographic Regions with Agent and Retrieval models
  6. 2024Customized large language models can outperform Community Notes in correcting misinformation
  7. 2025CANDY: Benchmarking LLMs' Limitations and Assistive Potential in Chinese Misinformation Fact-Checking
  8. 2025Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs