# VERISCORE: Evaluating the factuality of verifiable claims in long-form text generation

arXiv: https://arxiv.org/abs/2406.19276 · HTML: https://mlsanity.com/paper/2406.19276/

## Key facts

- Cited by 117 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 27 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents](https://mlsanity.com/paper/2404.10774/) (2024)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://mlsanity.com/paper/2307.13528/) (2023)
- [AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web](https://mlsanity.com/paper/2305.13117/) (2023)
- [WiCE: Real-World Entailment for Claims in Wikipedia](https://mlsanity.com/paper/2303.01432/) (2023)
- [A Critical Evaluation of Evaluations for Long-form Question Answering](https://mlsanity.com/paper/2305.18201/) (2023)
- [Complex Claim Verification with Evidence Retrieved in the Wild](https://mlsanity.com/paper/2305.11859/) (2023)
- [Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes](https://mlsanity.com/paper/2008.09094/) (2020)
- [FLAME: Factuality-Aware Alignment for Large Language Models](https://mlsanity.com/paper/2405.01525/) (2024)
- [Effective Large Language Model Adaptation for Improved Grounding and Citation Generation](https://mlsanity.com/paper/2311.09533/) (2023)
- [A Closer Look at Claim Decomposition](https://mlsanity.com/paper/2403.11903/) (2024)
- [Language Models Hallucinate, but May Excel at Fact Verification](https://mlsanity.com/paper/2310.14564/) (2023)
- [SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums](https://mlsanity.com/paper/1906.01727/) (2019)
- [The Role of Context in Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2104.07423/) (2021)
- [Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generations](https://mlsanity.com/paper/2402.05629/) (2024)

## Built on by (most cited first, 30 of 117)

- [OverThink: Slowdown Attacks on Reasoning LLMs](https://mlsanity.com/paper/2502.02542/) (2025)
- [HalluLens: LLM Hallucination Benchmark](https://mlsanity.com/paper/2504.17550/) (2025)
- [No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding](https://mlsanity.com/paper/2503.05061/) (2025)
- [SimpleQA Verified: A Reliable Factuality Benchmark to Measure Parametric Knowledge](https://mlsanity.com/paper/2509.07968/) (2025)
- [The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Input](https://mlsanity.com/paper/2501.03200/) (2025)
- [DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents](https://mlsanity.com/paper/2601.20975/) (2026)
- [Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?](https://mlsanity.com/paper/2411.02400/) (2024)
- [Measuring and Enhancing Trustworthiness of LLMs in RAG through Grounded Attributions and Learning to Refuse](https://mlsanity.com/paper/2409.11242/) (2024)
- [FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation](https://mlsanity.com/paper/2410.22257/) (2024)
- [DRAGged into Conflicts: Detecting and Addressing Conflicting Sources in Search-Augmented LLMs](https://mlsanity.com/paper/2506.08500/) (2025)
- [Claim Verification in the Age of Large Language Models: A Survey](https://mlsanity.com/paper/2408.14317/) (2024)
- [Atomic Calibration of LLMs in Long-Form Generations](https://mlsanity.com/paper/2410.13246/) (2024)
- [DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generation](https://mlsanity.com/paper/2412.13175/) (2024)
- [Conformal Linguistic Calibration: Trading-off between Factuality and Specificity](https://mlsanity.com/paper/2502.19110/) (2025)
- [Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2504.14891/) (2025)
- [Towards Effective Extraction and Evaluation of Factual Claims](https://mlsanity.com/paper/2502.10855/) (2025)
- [BLEUBERI: BLEU is a surprisingly effective reward for instruction following](https://mlsanity.com/paper/2505.11080/) (2025)
- [Core: Robust Factual Precision with Informative Sub-Claim Identification](https://mlsanity.com/paper/2407.03572/) (2024)
- [SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generation](https://mlsanity.com/paper/2508.15658/) (2025)
- [Beyond Factual Accuracy: Evaluating Coverage of Diverse Factual Information in Long-form Text Generation](https://mlsanity.com/paper/2501.03545/) (2025)
- [Multi-group Uncertainty Quantification for Long-form Text Generation](https://mlsanity.com/paper/2407.21057/) (2024)
- [Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifiers](https://mlsanity.com/paper/2506.13342/) (2025)
- [UNCLE: Benchmarking Uncertainty Expressions in Long-Form Generation](https://mlsanity.com/paper/2505.16922/) (2025)
- [Improving Factuality with Explicit Working Memory](https://mlsanity.com/paper/2412.18069/) (2024)
- [Optimizing Decomposition for Optimal Claim Verification](https://mlsanity.com/paper/2503.15354/) (2025)
- [Does quantization affect models' performance on long-context tasks?](https://mlsanity.com/paper/2505.20276/) (2025)
- [DeepFact: Co-Evolving Benchmarks and Agents for Deep Research Factuality](https://mlsanity.com/paper/2603.05912/) (2026)
- [MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration](https://mlsanity.com/paper/2503.15272/) (2025)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [VeriFastScore: Speeding up long-form factuality evaluation](https://mlsanity.com/paper/2505.16973/) (2025)

## Abstract (arXiv)

Existing metrics for evaluating the factuality of long-form text, such as FACTSCORE (Min et al., 2023) and SAFE (Wei et al., 2024), decompose an input text into "atomic claims" and verify each against a knowledge base like Wikipedia. These metrics are not suitable for most generation tasks because they assume that every claim is verifiable (i.e., can plausibly be proven true or false). We address this issue with VERISCORE, a metric for diverse long-form generation tasks that contain both verifiable and unverifiable content. VERISCORE can be effectively implemented with either closed or fine-tuned open-weight language models, and human evaluation confirms that VERISCORE's extracted claims are more sensible than those from competing methods across eight different long-form tasks. We use VERISCORE to evaluate generations from 16 different models across multiple long-form tasks and find that while GPT-4o is the best-performing model overall, open-weight models such as Mixtral-8x22 are closing the gap. We show that an LM's VERISCORE on one task (e.g., biography generation) does not necessarily correlate to its VERISCORE on a different task (e.g., long-form QA), highlighting the need for expanding factuality evaluation across tasks with varying fact density.