mlsanityData to 13 Oct 2026

VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation

arXiv 2406.19276 ↗First posted 27 Jun 2024cs.CLcited by 116

Not on the map. Why?

Abstract (arXiv)

Existing metrics for evaluating the factuality of long-form text, such as FACTSCORE (Min et al., 2023) and SAFE (Wei et al., 2024), decompose an input text into "atomic claims" and verify each against a knowledge base like Wikipedia. These metrics are not suitable for most generation tasks because they assume that every claim is verifiable (i.e., can plausibly be proven true or false). We address this issue with VERISCORE, a metric for diverse long-form generation tasks that contain both verifiable and unverifiable content. VERISCORE can be effectively implemented with either closed or fine-tuned open-weight language models, and human evaluation confirms that VERISCORE's extracted claims are more sensible than those from competing methods across eight different long-form tasks. We use VERISCORE to evaluate generations from 16 different models across multiple long-form tasks and find that while GPT-4o is the best-performing model overall, open-weight models such as Mixtral-8x22 are closing the gap. We show that an LM's VERISCORE on one task (e.g., biography generation) does not necessarily correlate to its VERISCORE on a different task (e.g., long-form QA), highlighting the need for expanding factuality evaluation across tasks with varying fact density.

Key facts. Cited by 116 papers in the map (data through 13 Oct 2026). Builds on 23 of its 46 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

23 of 46 references.

References · 23

  1. 2016Pointer Sentinel Mixture Modelscited by 3,047
  2. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,368
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,268
  4. 2018Hierarchical Neural Story Generationcited by 1,186
  5. 2023In-Context Retrieval-Augmented Language Modelscited by 699
  6. 2024OLMo: Accelerating the Science of Language Modelscited by 605
  7. 2020Fact or Fiction: Verifying Scientific Claimscited by 552
  8. 2023A Survey of Hallucination in Large Foundation Modelscited by 367
  9. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 312
  10. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 255
  11. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 248
  12. 2023AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Webcited by 144
11 more
  1. 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 119
  2. 2023A Critical Evaluation of Evaluations for Long-form Question Answeringcited by 111
  3. 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110
  4. 2020Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotescited by 101
  5. 2024FLAME: Factuality-Aware Alignment for Large Language Modelscited by 46
  6. 2023Effective Large Language Model Adaptation for Improved Grounding and Citation Generationcited by 44
  7. 2024A Closer Look at Claim Decompositioncited by 40
  8. 2023Language Models Hallucinate, but May Excel at Fact Verificationcited by 36
  9. 2019SemEval-2019 Task 8: Fact Checking in Community Question Answering Forumscited by 18
  10. 2021The Role of Context in Detecting Previously Fact-Checked Claimscited by 17
  11. 2024Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generationscited by 14

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 10; 2025: 54; 2026: 52.

116 papers.

Sort:

By area:

  1. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 95
  2. 2025HalluLens: LLM Hallucination Benchmarkcited by 90
  3. 2025The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Inputcited by 51
  4. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 50
  5. 2025SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledgecited by 46
  6. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agentscited by 44
  7. 2024Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?cited by 34
  8. 2024Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refusecited by 33
  9. 2024Claim Verification in the Age of Large Language Models: A Surveycited by 24
  10. 2024Atomic Calibration of LLMs in Long-Form Generationscited by 24
  11. 2024FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluationcited by 24
  12. 2024DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generationcited by 22
38 more
  1. 2025DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMscited by 22
  2. 2025Conformal Linguistic Calibration: Trading-off between Factuality and Specificitycited by 20
  3. 2025Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Surveycited by 20
  4. 2025Towards Effective Extraction and Evaluation of Factual Claimscited by 19
  5. 2025BLEUBERI: BLEU is a surprisingly effective reward for instruction followingcited by 17
  6. 2024Core: Robust Factual Precision with Informative Sub-Claim Identificationcited by 15
  7. 2025Beyond Factual Accuracy: Evaluating Coverage of Diverse Factual Information in Long-form Text Generationcited by 15
  8. 2025SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generationcited by 15
  9. 2024Multi-group Uncertainty Quantification for Long-form Text Generationcited by 13
  10. 2025Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifierscited by 13
  11. 2025UNCLE: Benchmarking Uncertainty Expressions in Long-Form Generationcited by 11
  12. 2024Improving Factuality with Explicit Working Memorycited by 10
  13. 2025Optimizing Decomposition for Optimal Claim Verificationcited by 10
  14. 2025Does quantization affect models' performance on long-context tasks?cited by 10
  15. 2026DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factualitycited by 10
  16. 2025MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaborationcited by 9
  17. 2025VeriFastScore: Speeding up long-form factuality evaluationcited by 8
  18. 2025FaStfact: Faster, Stronger Long-Form Factuality Evaluations in LLMscited by 8
  19. 2025Can LLMs Automate Fact-Checking Article Writing?cited by 7
  20. 2025Towards Long Context Hallucination Detectioncited by 7
  21. 2025DesignPref: Capturing Personal Preferences in Visual Design Generationcited by 7
  22. 2026To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Callingcited by 7
  23. 2026Autonomous Research Agents: A Survey of AI Scientists and the Verification Gapcited by 7
  24. 2025How Does Response Length Affect Long-Form Factualitycited by 6
  25. 2025All Claims Are Equal, but Some Claims Are More Equal Than Others: Importance-Sensitive Factuality Evaluation of LLM Generationscited by 6
  26. 2025MedScore: Generalizable Factuality Evaluation of Free-Form Medical Answers by Domain-adapted Claim Decomposition and Verificationcited by 5
  27. 2025FACTORY: A Challenging Human-Verified Prompt Set for Long-Form Factualitycited by 5
  28. 2025If We May De-Presuppose: Robustly Verifying Claims through Presupposition-Free Question Decompositioncited by 5
  29. 2025Inflated Excellence or True Performance? Rethinking Medical Diagnostic Benchmarks with Dynamic Evaluationcited by 5
  30. 2025Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinationscited by 5
  31. 2025Explanatory Summarization with Discourse-Driven Planningcited by 4
  32. 2025Long-Form Information Alignment Evaluation Beyond Atomic Factscited by 4
  33. 2025Detecting Corpus-Level Knowledge Inconsistencies in Wikipedia with Large Language Modelscited by 4
  34. 2025MAD-Fact: A Multi-Agent Debate Framework for Long-Form Factuality Evaluation in LLMscited by 4
  35. 2025MR-Align: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Modelscited by 4
  36. 2026MovieRecapsQA: A Multimodal Open-Ended Video Question-Answering Benchmarkcited by 4
  37. 2026Beyond Single-shot Writing: Deep Research Agents are Unreliable at Multi-turn Report Revisioncited by 4
  38. 2026Beyond Precision: Importance-Aware Recall for Factuality Evaluation in Long-Form LLM Generationcited by 4

50 most cited of 116 All 116 (JSON)