mlsanityData to 7 Oct 2026

MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents

arXiv 2404.10774 ↗First posted 16 Apr 2024cs.CL cs.AIcited by 251

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Recognizing if LLM output can be grounded in evidence is central to many tasks in NLP: retrieval-augmented generation, summarization, document-grounded dialogue, and more. Current approaches to this kind of fact-checking are based on verifying each piece of a model generation against potential evidence using an LLM. However, this process can be very computationally expensive, requiring many calls to a model to check a single response. In this work, we show how to build small fact-checking models that have GPT-4-level performance but for 400x lower cost. We do this by constructing synthetic training data with GPT-4, which involves creating realistic yet challenging instances of factual errors via a structured generation procedure. Training on this data teaches models to check each fact in the claim and recognize synthesis of information across sentences. For evaluation, we unify datasets from recent work on fact-checking and grounding LLM generations into a new benchmark, LLM-AggreFact. Our best system MiniCheck-FT5 (770M parameters) outperforms all systems of comparable size and reaches GPT-4 accuracy. We release LLM-AggreFact, code for data synthesis, and models.

Key facts. Cited by 251 papers in the map (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 55 of its 76 references that are in the map. First posted on arXiv 16 Apr 2024.

Builds on

55 of 76 references.

References · 55

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  4. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  6. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  7. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  8. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  9. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  11. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  12. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
38 more
  1. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  2. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  3. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  4. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  5. 2019ELI5: Long Form Question Answeringcited by 567
  6. 2023Benchmarking Large Language Models for News Summarizationcited by 444
  7. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 402
  8. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  9. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  10. 2023How Language Model Hallucinations Can Snowballcited by 310
  11. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  12. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  13. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  14. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 285
  15. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  16. 2021QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarizationcited by 233
  17. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  18. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  19. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  20. 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 148
  21. 2021Annotating and Modeling Fine-grained Factuality in Summarizationcited by 131
  22. 2021Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarizationcited by 128
  23. 2020Evaluating Factuality in Generation with Dependency-level Entailmentcited by 122
  24. 2021CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarizationcited by 120
  25. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
  26. 2021MediaSum: A Large-scale Media Interview Dataset for Dialogue Summarizationcited by 104
  27. 2022Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectorscited by 103
  28. 2021Decontextualization: Making Sentences Stand-Alonecited by 101
  29. 2023MeetingBank: A Benchmark Dataset for Meeting Summarizationcited by 81
  30. 2021DocNLI: A Large-scale Dataset for Document-level Natural Language Inferencecited by 75
  31. 2023ChatGPT as a Factual Inconsistency Evaluator for Text Summarizationcited by 75
  32. 2024TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarizationcited by 66
  33. 2020What Have We Achieved on Text Summarization?cited by 65
  34. 2024A Chain-of-Thought Is as Strong as Its Weakest Link: A Benchmark for Verifiers of Reasoning Chainscited by 63
  35. 2023Self-Verification Improves Few-Shot Clinical Information Extractioncited by 52
  36. 2023Understanding Retrieval Augmentation for Long-Form Question Answeringcited by 45
  37. 2023Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)cited by 40
  38. 2021AmbiFC: Fact-Checking Ambiguous Claims with Evidencecited by 38

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 50; 2025: 100; 2026: 101.

251 papers.

Sort:

By area:

  1. 2024VERISCORE: Evaluating the factuality of verifiable claims in long-form text generationcited by 114
  2. 2024Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systemscited by 87
  3. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 72
  4. 2025The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Inputcited by 49
  5. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agentscited by 44
  6. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  7. 2024Molecular Facts: Desiderata for Decontextualization in LLM Fact Verificationcited by 42
  8. 2025SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Paperscited by 42
  9. 2024Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?cited by 34
  10. 2025PrivaCI-Bench: Evaluating Privacy with Contextual Integrity and Legal Compliancecited by 34
  11. 2024Direct Judgement Preference Optimizationcited by 33
  12. 2024LMUnit: Fine-grained Evaluation with Natural Language Unit Testscited by 32
38 more
  1. 2024Learning to Refine with Fine-Grained Natural Language Feedbackcited by 27
  2. 2025Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievalscited by 27
  3. 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Datacited by 26
  4. 2024Learning to Summarize from LLM-generated Feedbackcited by 24
  5. 2024FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMscited by 24
  6. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 23
  7. 2024On Positional Bias of Faithfulness for Long-form Summarizationcited by 22
  8. 2024DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generationcited by 22
  9. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 22
  10. 2024UniSumEval: Towards Unified, Fine-Grained, Multi-Dimensional Summarization Evaluation for LLMscited by 21
  11. 2025DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMscited by 21
  12. 2025Shrinking the Generation-Verification Gap with Weak Verifierscited by 21
  13. 2025Conformal Linguistic Calibration: Trading-off between Factuality and Specificitycited by 20
  14. 2026AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversationscited by 20
  15. 2024FinDVer: Explainable Claim Verification over Long and Hybrid-Content Financial Documentscited by 19
  16. 2025Benchmarking LLM Faithfulness in RAG with Evolving Leaderboardscited by 19
  17. 2024CodeUpdateArena: Benchmarking Knowledge Editing on API Updatescited by 18
  18. 2024ECon: On the Detection and Resolution of Evidence Conflictscited by 18
  19. 2025Towards Effective Extraction and Evaluation of Factual Claimscited by 18
  20. 2024From Single to Multi: How LLMs Hallucinate in Multi-Document Summarizationcited by 17
  21. 2024Core: Robust Factual Precision with Informative Sub-Claim Identificationcited by 15
  22. 2024Improving Model Factuality with Fine-grained Critique-based Evaluatorcited by 15
  23. 2024Search Engines in an AI Era: The False Promise of Factual and Verifiable Source-Cited Responsescited by 15
  24. 2024Logical Consistency of Large Language Models in Fact-checkingcited by 15
  25. 2025QFFT, Question-Free Fine-Tuning for Adaptive Reasoningcited by 15
  26. 2024Halu-J: Critique-Based Hallucination Judgecited by 14
  27. 2025FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Datacited by 14
  28. 2025The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factualitycited by 14
  29. 2026JEV-as-a-Judge: Accept When Confident, Escalate When Unsurecited by 14
  30. 2024Localizing Factual Inconsistencies in Attributable Text Generationcited by 13
  31. 2025Hallucination to Truth: A Review of Fact-Checking and Factuality Evaluation in Large Language Modelscited by 13
  32. 2025GraphCheck: Breaking Long-Term Text Barriers with Extracted Knowledge Graph-Powered Fact-Checkingcited by 12
  33. 2025Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifierscited by 12
  34. 2025Co-Sight: Enhancing LLM-Based Agents via Conflict-Aware Meta-Verification and Trustworthy Reasoning with Structured Factscited by 12
  35. 2025Behavioral Analysis of Information Salience in Large Language Modelscited by 11
  36. 2025Toward Holistic Evaluation of Recommender Systems Powered by Generative Modelscited by 11
  37. 2024Attribute or Abstain: Large Language Models as Long Document Assistantscited by 10
  38. 2024Understanding Synthetic Context Extension via Retrieval Headscited by 10

50 most cited of 251 All 251 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Fact-checking AI-generated news reports: Can LLMs catch their own lies?
  2. 2023The Perils & Promises of Fact-checking with Large Language Models
  3. 2024Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output
  4. 2023LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
  5. 2026Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checking
  6. 2025Resolving Conflicting Evidence in Automated Fact-Checking: A Study on Retrieval-Augmented LLMs
  7. 2025Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs
  8. 2024Learning to Verify Summary Facts with Fine-Grained LLM Feedback