# FIRE: Fact-checking with Iterative Retrieval and Verification

arXiv: https://arxiv.org/abs/2411.00784 · HTML: https://mlsanity.com/paper/2411.00784/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 47 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [A Survey of Confidence Estimation and Calibration in Large Language Models](https://mlsanity.com/paper/2311.08298/) (2023)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://mlsanity.com/paper/2307.13528/) (2023)
- [Chain-of-Thought Reasoning Without Prompting](https://mlsanity.com/paper/2402.10200/) (2024)
- [The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey](https://mlsanity.com/paper/2404.11584/) (2024)
- [Long-form factuality in large language models](https://mlsanity.com/paper/2403.18802/) (2024)
- [Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers](https://mlsanity.com/paper/2311.09000/) (2023)
- [The Next Chapter: A Study of Large Language Models in Storytelling](https://mlsanity.com/paper/2301.09790/) (2023)
- [What Are Tools Anyway? A Survey from the Language Model Perspective](https://mlsanity.com/paper/2403.15452/) (2024)
- [Multimodal Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2305.13507/) (2023)
- [OpenFactCheck: A Unified Framework for Factuality Evaluation of LLMs](https://mlsanity.com/paper/2408.11832/) (2024)
- [Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector](https://mlsanity.com/paper/2406.11277/) (2024)
- [Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework](https://mlsanity.com/paper/2406.03075/) (2024)
- [Loki: An Open-Source Tool for Fact Verification](https://mlsanity.com/paper/2410.01794/) (2024)
- [DeltaScore: Fine-Grained Story Evaluation with Perturbations](https://mlsanity.com/paper/2303.08991/) (2023)

## Built on by (most cited first, 30 of 39)

- [Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Models](https://mlsanity.com/paper/2508.03860/) (2025)
- [DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factuality](https://mlsanity.com/paper/2603.05912/) (2026)
- [A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QA](https://mlsanity.com/paper/2509.21199/) (2025)
- [FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMs](https://mlsanity.com/paper/2510.12839/) (2025)
- [Can LLMs Automate Fact-Checking Article Writing?](https://mlsanity.com/paper/2503.17684/) (2025)
- [COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations](https://mlsanity.com/paper/2510.24810/) (2025)
- [UrduFactCheck: An Agentic Fact-Checking Framework for Urdu with Evidence Boosting and Benchmarking](https://mlsanity.com/paper/2505.15063/) (2025)
- [MAD-Fact: A Multi-Agent Debate Framework for Long-Form Factuality Evaluation in LLMs](https://mlsanity.com/paper/2510.22967/) (2025)
- [Addressing Moral Uncertainty using Large Language Models for Ethical Decision-Making](https://mlsanity.com/paper/2503.05724/) (2025)
- [EMULATE: A Multi-Agent Framework for Determining the Veracity of Atomic Claims by Emulating Human Actions](https://mlsanity.com/paper/2505.16576/) (2025)
- [Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking](https://mlsanity.com/paper/2603.00267/) (2026)
- [Fact in Fragments: Deconstructing Complex Claims via LLM-based Atomic Fact Extraction and Verification](https://mlsanity.com/paper/2506.07446/) (2025)
- [Recon, Answer, Verify: Agents in Search of Truth](https://mlsanity.com/paper/2507.03671/) (2025)
- [A New Dataset and Benchmark for Grounding Multimodal Misinformation](https://mlsanity.com/paper/2509.08008/) (2025)
- [A Quantitative Study of Sustained Focus in Large Language Models via Repetitive Deterministic Prediction Tasks](https://mlsanity.com/paper/2511.00763/) (2025)
- [Hybrid Fact-Checking that Integrates Knowledge Graphs, Large Language Models, and Search-Based Retrieval Agents Improves Interpretable Claim Verification](https://mlsanity.com/paper/2511.03217/) (2025)
- [The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systems](https://mlsanity.com/paper/2602.10886/) (2026)
- [Co-FactChecker: A Framework for Human-AI Collaborative Claim Verification Using Large Reasoning Models](https://mlsanity.com/paper/2604.13706/) (2026)
- [Debating the Unspoken: Role-Anchored Multi-Agent Reasoning for Half-Truth Detection](https://mlsanity.com/paper/2604.19005/) (2026)
- [GSAR: Typed Grounding for Hallucination Detection and Recovery in Multi-Agent LLMs](https://mlsanity.com/paper/2604.23366/) (2026)
- [R2VC: Modular Fact-Checking with Retrieval, Verification, and Confidence Calibration](https://mlsanity.com/paper/2609.11955/) (2026)
- [Can a Small Model Learn to Look Before It Leaps? Dynamic Learning and Proactive Correction for Hallucination Detection](https://mlsanity.com/paper/2511.05854/) (2025)
- [TrackList: Tracing Back Query Linguistic Diversity for Head and Tail Knowledge in Open Large Language Models](https://mlsanity.com/paper/2511.21006/) (2025)
- [Fact-Checking with Large Language Models via Probabilistic Certainty and Consistency](https://mlsanity.com/paper/2601.02574/) (2026)
- [MERMAID: Memory-Enhanced Retrieval and Reasoning with Multi-Agent Iterative Knowledge Grounding for Veracity Assessment](https://mlsanity.com/paper/2601.22361/) (2026)
- [Is a Picture Worth a Thousand Words? Adaptive Multimodal Fact-Checking with Visual Evidence Necessity](https://mlsanity.com/paper/2604.04692/) (2026)
- [ArbGraph: Conflict-Aware Evidence Arbitration for Reliable Long-Form Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.18362/) (2026)
- [Evergreen: Efficient Claim Verification for Semantic Aggregates](https://mlsanity.com/paper/2604.26180/) (2026)
- [Position: Uncertainty Quantification in LLMs is Just Unsupervised Clustering](https://mlsanity.com/paper/2605.19220/) (2026)
- [Generating Biomedical Fact-Checking Reports with RL-Enhanced Agentic Search](https://mlsanity.com/paper/2608.23811/) (2026)

## Abstract (arXiv)

Fact-checking long-form text is challenging, and it is therefore common practice to break it down into multiple atomic claims. The typical approach to fact-checking these atomic claims involves retrieving a fixed number of pieces of evidence, followed by a verification step. However, this method is usually not cost-effective, as it underutilizes the verification model's internal knowledge of the claim and fails to replicate the iterative reasoning process in human search strategies. To address these limitations, we propose FIRE, a novel agent-based framework that integrates evidence retrieval and claim verification in an iterative manner. Specifically, FIRE employs a unified mechanism to decide whether to provide a final answer or generate a subsequent search query, based on its confidence in the current judgment. We compare FIRE with other strong fact-checking frameworks and find that it achieves slightly better performance while reducing large language model (LLM) costs by an average of 7.6 times and search costs by 16.5 times. These results indicate that FIRE holds promise for application in large-scale fact-checking operations. Our code is available at https://github.com/mbzuai-nlp/fire.git.