mlsanityData to 7 Oct 2026

Large Language Models Require Curated Context for Reliable Political Fact-Checking -- Even with Reasoning and Web Search

arXiv 2511.18749 ↗First posted 24 Nov 2025cs.CL cs.CY cs.IRcited by 2

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models (LLMs) have raised hopes for automated end-to-end fact-checking, but prior studies report mixed results. As mainstream chatbots increasingly ship with reasoning capabilities and web search tools -- and millions of users already rely on them for verification -- rigorous evaluation is urgent. We evaluate 15 recent LLMs from OpenAI, Google, Meta, and DeepSeek on more than 6,000 claims fact-checked by PolitiFact, comparing standard models with reasoning- and web-search variants. Standard models perform poorly, reasoning offers minimal benefits, and web search provides only moderate gains, despite fact-checks being available on the web. In contrast, a curated RAG system using PolitiFact summaries improved macro F1 by 233% on average across model variants. These findings suggest that giving models access to curated high-quality context is a promising path for automated fact-checking.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 25 of its 66 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

25 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Fact-Checking Complex Claims with Program-Guided Reasoning22 May

Other references · 24

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  3. 2023Active Retrieval Augmented Generationcited by 506
  4. 2017"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detectioncited by 361
  5. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  6. 2019MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claimscited by 169
  7. 2023Factuality Challenges in the Era of Large Language Modelscited by 125
  8. 2018Combining Fact Extraction and Verification with Neural Semantic Matching Networkscited by 120
  9. 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110
  10. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  11. 2020Explainable Automated Fact-Checking: A Surveycited by 83
  12. 2018RumourEval 2019: Determining Rumour Veracity and Support for Rumourscited by 74
12 more
  1. 2022CREPE: Open-Domain Question Answering with False Presuppositionscited by 70
  2. 2019BERT for Evidence Retrieval and Claim Verificationcited by 57
  3. 2021Automated Fact-Checking: A Surveycited by 55
  4. 2020Language Models as Fact Checkers?cited by 51
  5. 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
  6. 2024On the Relationship between Truth and Political Bias in Language Modelscited by 31
  7. 2023Fact-checking information from large language models can decrease headline discernmentcited by 22
  8. 2024Supernotes: Driving Consensus in Crowd-Sourced Fact-Checkingcited by 22
  9. 2025News Source Citing Patterns in AI Search Systemscited by 9
  10. 2022NELA-Local: A Dataset of U.S. Local News Articles for the Study of County-level News Ecosystemscited by 6
  11. 2025Evaluating open-source Large Language Models for automated fact-checkingcited by 6
  12. 2025DomainDemo: a dataset of domain-sharing activities among different demographic groups on Twittercited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026GitSearch: Enhancing Community Notes Generation with Gap-Informed Targeted Searchcited by 2
  2. 2025Evaluating Reliability Asymmetries in Chinese Factual Search and AI Answerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025How LLMs Fail to Support Fact-Checking
  2. 2024RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Models
  3. 2025Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs
  4. 2025Understanding Inequality of LLM Fact-Checking over Geographic Regions with Agent and Retrieval models
  5. 2025Recon, Answer, Verify: Agents in Search of Truth
  6. 2026PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts
  7. 2023The Perils & Promises of Fact-checking with Large Language Models
  8. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models