mlsanityData to 14 Oct 2026

Assessing The Factual Accuracy of Generated Text

arXiv 1905.13322 ↗First posted 30 May 2019cs.CLcited by 103

Not on the map. Why?

Abstract (arXiv)

We propose a model-based metric to estimate the factual accuracy of generated text that is complementary to typical scoring schemes like ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and BLEU (Bilingual Evaluation Understudy). We introduce and release a new large-scale dataset based on Wikipedia and Wikidata to train relation classifiers and end-to-end fact extraction models. The end-to-end models are shown to be able to extract complete sets of facts from datasets with full pages of text. We then analyse multiple models that estimate factual accuracy on a Wikipedia text summarization task, and show their efficacy compared to ROUGE and other model-free variants by conducting a human evaluation study.

Key facts. Cited by 103 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 20; 2023: 28; 2024: 22; 2025: 13; 2026: 4.

103 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,576
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  4. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
  5. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
  6. 2021Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metricscited by 287
  7. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
  8. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  9. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 197
  10. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 138
  11. 2021Q2: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answeringcited by 132
  12. 2022Evaluating the Factual Consistency of Large Language Models Through News Summarizationcited by 92
38 more
  1. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 88
  2. 2023TrueTeacher: Learning Factual Consistency Evaluation with Large Language Modelscited by 83
  3. 2021The Factual Inconsistency Problem in Abstractive Text Summarization: A Surveycited by 79
  4. 2024Safeguarding Large Language Models: A Surveycited by 78
  5. 2023ChatGPT as a Factual Inconsistency Evaluator for Text Summarizationcited by 75
  6. 2022An Empirical Survey on Long Document Summarization: Datasets, Models and Metricscited by 74
  7. 2023WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipediacited by 67
  8. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  9. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
  10. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 44
  11. 2023Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questionscited by 42
  12. 2022FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representationscited by 41
  13. 2024A Closer Look at Claim Decompositioncited by 41
  14. 2022Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infillingcited by 38
  15. 2021Factual Consistency Evaluation for Text Summarization via Counterfactual Estimationcited by 37
  16. 2021A Study of Automatic Metrics for the Evaluation of Natural Language Explanationscited by 34
  17. 2022Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarizationcited by 34
  18. 2024Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selectioncited by 33
  19. 2024Measuring and Reducing LLM Hallucination without Gold-Standard Answerscited by 31
  20. 2023MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarizationcited by 30
  21. 2023WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpuscited by 29
  22. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  23. 2022How Far are We from Robust Long Abstractive Summarization?cited by 25
  24. 2021Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric Viewcited by 24
  25. 2025VeriSafe Agent: Safeguarding Mobile GUI Agent via Logic-based Action Verificationcited by 22
  26. 2023Zero-shot Faithfulness Evaluation for Text Summarization with Foundation Language Modelcited by 21
  27. 2021TODSum: Task-Oriented Dialogue Summarization with State Trackingcited by 19
  28. 2025Towards Effective Extraction and Evaluation of Factual Claimscited by 19
  29. 2024Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challengescited by 18
  30. 2024Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checkingcited by 18
  31. 2024Related Work and Citation Text Generation: A Surveycited by 17
  32. 2021Evaluating the Tradeoff Between Abstractiveness and Factuality in Abstractive Summarizationcited by 16
  33. 2023Measuring and Modifying Factual Knowledge in Large Language Modelscited by 16
  34. 2024FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Documentcited by 15
  35. 2023A Meta-Evaluation of Faithfulness Metrics for Long-Form Hospital-Course Summarizationcited by 14
  36. 2023Challenges and Remedies to Privacy and Security in AIGC: Exploring the Potential of Privacy Computing, Blockchain, and Beyondcited by 14
  37. 2023Reliability Check: An Analysis of GPT-3's Response to Sensitive Topics and Prompt Wordingcited by 13
  38. 2022Table-based Fact Verification with Self-adaptive Mixture of Expertscited by 12

50 most cited of 103 All 103 (JSON)