mlsanityData to 14 Oct 2026

Verify with Caution: The Pitfalls of Relying on Imperfect Factuality Metrics

arXiv 2501.14883 ↗First posted 24 Jan 2025cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Improvements in large language models have led to increasing optimism that they can serve as reliable evaluators of natural language generation outputs. In this paper, we challenge this optimism by thoroughly re-evaluating five state-of-the-art factuality metrics on a collection of 11 datasets for summarization, retrieval-augmented generation, and question answering. We find that these evaluators are inconsistent with each other and often misestimate system-level performance, both of which can lead to a variety of pitfalls. We further show that these metrics exhibit biases against highly paraphrased outputs and outputs that draw upon faraway parts of the source documents. We urge users of these factuality metrics to proceed with caution and manually validate the reliability of these metrics in their domain of interest before proceeding.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 24 Jan 2025.

Builds on

19 of 31 references.

References · 19

  1. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  2. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  3. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 283
  4. 2024MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documentscited by 257
  5. 2021Measuring Attribution in Natural Language Generation Modelscited by 256
  6. 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 163
  7. 2020Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metricscited by 158
  8. 2023Complex Claim Verification with Evidence Retrieved in the Wildcited by 110
  9. 2023LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarizationcited by 96
  10. 2024Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluationcited by 74
  11. 2024TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarizationcited by 66
  12. 2023PURR: Efficiently Editing Language Model Hallucinations by Denoising Language Model Corruptionscited by 53
7 more
  1. 2021Perturbation CheckLists for Evaluating NLG Evaluation Metricscited by 50
  2. 2023Understanding Retrieval Augmentation for Long-Form Question Answeringcited by 45
  3. 2024AttributionBench: How Hard is Automatic Attribution Evaluation?cited by 26
  4. 2024On Positional Bias of Faithfulness for Long-form Summarizationcited by 22
  5. 2023NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklistcited by 16
  6. 2024Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluationcited by 11
  7. 2024Schroedinger's Threshold: When the AUC doesn't predict Accuracycited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2025How to Correctly Report LLM-as-a-Judge Evaluationscited by 30
  2. 2025Verifying the Verifiers: Unveiling Pitfalls and Potentials in Fact Verifierscited by 13
  3. 2025Evaluating Evaluation Metrics -- The Mirage of Hallucination Detectioncited by 6
  4. 2025All Claims Are Equal, but Some Claims Are More Equal Than Others: Importance-Sensitive Factuality Evaluation of LLM Generationscited by 6
  5. 2026How College Students Use AI to Navigate Course Readings: Evidence from an Eight-Week Studycited by 0
  6. 2026Does Finetuning with Scientific Data Increase Hallucinations? A Multi-domain Factuality Evaluation of LLMscited by 0
  7. 2026Can We Trust the Judges? Validation of Factuality Evaluation Methods via Answer Perturbationcited by 0
  8. 2026RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
  2. 2024Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknown
  3. 2024Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generations
  4. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
  5. 2024Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output
  6. 2022On Improving Summarization Factual Consistency from Natural Language Feedback
  7. 2024UFO: a Unified and Flexible Framework for Evaluating Factuality of Large Language Models
  8. 2024Long-form factuality in large language models