mlsanityData to 7 Oct 2026

Fact-Checking Complex Claims with Program-Guided Reasoning

arXiv 2305.12744 ↗First posted 22 May 2023cs.CL cs.AIcited by 146

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Fact-checking real-world claims often requires collecting multiple pieces of evidence and applying complex multi-step reasoning. In this paper, we present Program-Guided Fact-Checking (ProgramFC), a novel fact-checking model that decomposes complex claims into simpler sub-tasks that can be solved using a shared library of specialized functions. We first leverage the in-context learning ability of large language models to generate reasoning programs to guide the verification process. Afterward, we execute the program by delegating each sub-task to the corresponding sub-task handler. This process makes our model both explanatory and data-efficient, providing clear explanations of its reasoning process and requiring minimal training data. We evaluate ProgramFC on two challenging fact-checking datasets and show that it outperforms seven fact-checking baselines across different settings of evidence availability, with explicit output programs that benefit human debugging. Our codes and data are publicly available at https://github.com/mbzuai-nlp/ProgramFC.

Key facts. Cited by 146 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 57 of its 79 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

57 of 79 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022PAL: Program-aided Language Models18 Nov
  2. 2022Measuring and Narrowing the Compositionality Gap in Language Models7 Oct

Other references · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  12. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
38 more
  1. 2020Longformer: The Long-Document Transformercited by 3,362
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  5. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  6. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  7. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  8. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  9. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  10. 2021A Survey on Automated Fact-Checkingcited by 376
  11. 2017"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detectioncited by 361
  12. 2022Binding Language Models in Symbolic Languagescited by 224
  13. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 222
  14. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 209
  15. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  16. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 199
  17. 2019MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claimscited by 169
  18. 2021FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured informationcited by 166
  19. 2021Automated Fact-Checking for Assisting Human Fact-Checkerscited by 144
  20. 2020Generating Fact Checking Explanationscited by 132
  21. 2018Combining Fact Extraction and Verification with Neural Semantic Matching Networkscited by 120
  22. 2020GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Mediacited by 108
  23. 2019Fine-grained Fact Verification with Kernel Graph Attention Networkcited by 105
  24. 2019GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verificationcited by 100
  25. 2021COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemiccited by 100
  26. 2020FANG: Leveraging Social Context for Fake News Detection Using Graph Representationcited by 98
  27. 2018Automated Fact Checking: Task formulations, methods and future directionscited by 96
  28. 2022SciFact-Open: Towards open-domain scientific claim verificationcited by 94
  29. 2021X-FACT: A New Benchmark Dataset for Multilingual Fact Checkingcited by 90
  30. 2019Reasoning Over Semantic-Level Graph for Fact Checkingcited by 69
  31. 2022Generating Literal and Implied Subquestions to Fact-check Complex Claimscited by 66
  32. 2021Towards Few-Shot Fact-Checking via Perplexitycited by 63
  33. 2022Generating Scientific Claims for Zero-Shot Scientific Fact Checkingcited by 59
  34. 2019BERT for Evidence Retrieval and Claim Verificationcited by 57
  35. 2021ProoFVer: Natural Logic Theorem Proving for Fact Verificationcited by 52
  36. 2020Language Models as Fact Checkers?cited by 51
  37. 2022Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformationcited by 45
  38. 2021Zero-shot Fact Verification by Claim Generationcited by 42

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 55; 2025: 44; 2026: 30.

146 papers.

Sort:

By area:

  1. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  2. 2024Tool Learning with Large Language Models: A Surveycited by 221
  3. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  4. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  5. 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsetscited by 122
  6. 2023AutoMix: Automatically Mixing Language Modelscited by 116
  7. 2023Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attackscited by 69
  8. 2023Do Large Language Models Know about Facts?cited by 49
  9. 2025The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Inputcited by 49
  10. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
  11. 2025SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledgecited by 44
  12. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
38 more
  1. 2023Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generatorscited by 42
  2. 2023Exploring the Potential of Large Language Models in Computational Argumentationcited by 42
  3. 2024DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Expertscited by 42
  4. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  5. 2023Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Modelscited by 37
  6. 2023EX-FEVER: A Dataset for Multi-hop Explainable Fact Verificationcited by 36
  7. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  8. 2024AKEW: Assessing Knowledge Editing in the Wildcited by 35
  9. 2024FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Modelcited by 34
  10. 2024Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?cited by 34
  11. 2024Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questionscited by 30
  12. 2024JustiLM: Few-shot Justification Generation for Explainable Fact-Checking of Real-world Claimscited by 29
  13. 2025Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checkingcited by 29
  14. 2023QACHECK: A Demonstration System for Question-Guided Multi-Hop Fact-Checkingcited by 26
  15. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 23
  16. 2024QuanTemp: A real-world open-domain benchmark for fact-checking numerical claimscited by 22
  17. 2023Towards the TopMost: A Topic Modeling System Toolkitcited by 21
  18. 2024Generative Large Language Models in Automated Fact-Checking: A Surveycited by 19
  19. 2025FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Modelscited by 19
  20. 2024A Survey on Neural Topic Models: Methods, Applications, and Challengescited by 18
  21. 2023From Chaos to Clarity: Claim Normalization to Empower Fact-Checkingcited by 17
  22. 2025AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Webcited by 17
  23. 2024Re-Search for The Truth: Multi-round Retrieval-augmented Large Language Models are Strong Fake News Detectorscited by 16
  24. 2024Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusioncited by 16
  25. 2025Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMscited by 16
  26. 2024MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Modelscited by 15
  27. 2024Automated Justification Production for Claim Veracity in Fact Checking: A Survey on Architectures and Approachescited by 15
  28. 2024Reinforcement Retrieval Leveraging Fine-grained Feedback for Fact Checking News Claims with Black-Box LLMcited by 14
  29. 2025COVE: COntext and VEracity prediction for out-of-context imagescited by 14
  30. 2024Multimodal Large Language Models to Support Real-World Fact-Checkingcited by 13
  31. 2024Piecing It All Together: Verifying Multi-Hop Multimodal Claimscited by 12
  32. 2024FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Modelscited by 11
  33. 2024Comparing Knowledge Sources for Open-Domain Scientific Claim Verificationcited by 10
  34. 2024TELLER: A Trustworthy Framework for Explainable, Generalizable and Controllable Fake News Detectioncited by 10
  35. 2024TART: An Open-Source Tool-Augmented Framework for Explainable Table-based Reasoningcited by 10
  36. 2024Are LLMs Good Zero-Shot Fallacy Classifiers?cited by 10
  37. 2024FactLens: Benchmarking Fine-Grained Fact Verificationcited by 10
  38. 2024Multimodal Fact-Checking with Vision Language Models: A Probing Classifier based Solution with Embedding Strategiescited by 10

50 most cited of 146 All 146 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025BOOST: Bootstrapping Strategy-Driven Reasoning Programs for Program-Guided Fact-Checking
  2. 2025Think Right, Not More: Test-Time Scaling for Numerical Claim Verification
  3. 2024CHECKWHY: Causal Fact Verification via Argument Structure
  4. 2023Evidence-based Interpretable Open-domain Fact-checking with Large Language Models
  5. 2025Fact in Fragments: Deconstructing Complex Claims via LLM-based Atomic Fact Extraction and Verification
  6. 2025Hybrid Fact-Checking that Integrates Knowledge Graphs, Large Language Models, and Search-Based Retrieval Agents Improves Interpretable Claim Verification
  7. 2026What Do Claim Verification Datasets Actually Test? A Reasoning Trace Analysis
  8. 2024Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables