mlsanityData to 14 Oct 2026

A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems

arXiv 2011.03992 ↗First posted 8 Nov 2020cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Most Natural Language Generation systems need to produce accurate texts. We propose a methodology for high-quality human evaluation of the accuracy of generated texts, which is intended to serve as a gold-standard for accuracy evaluations of data-to-text systems. We use our methodology to evaluate the accuracy of computer generated basketball summaries. We then show how our gold standard evaluation can be used to validate automated metrics

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 6; 2023: 6; 2024: 8; 2025: 6; 2026: 3.

34 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  2. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  3. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 155
  4. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 90
  5. 2023A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detectioncited by 42
  6. 2021Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generationcited by 29
  7. 2024Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practicescited by 20
  8. 2021Underreporting of errors in NLG output, and what to do about itcited by 19
  9. 2023Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignmentcited by 19
  10. 2022Data-to-text Generation with Variational Sequential Planningcited by 17
  11. 2024Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generationcited by 16
  12. 2023Quantifying and Attributing the Hallucination of Large Language Models via Association Analysiscited by 15
22 more
  1. 2024Face4RAG: Factual Consistency Evaluation for Retrieval Augmented Generation in Chinesecited by 10
  2. 2021Generation Challenges: Results of the Accuracy Evaluation Shared Taskcited by 9
  3. 2022Generating Full Length Wikipedia Biographies: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographiescited by 7
  4. 2025LLMs as Span Annotators: A Comparative Study of LLMs and Humanscited by 7
  5. 2025EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Modelscited by 7
  6. 2023HistAlign: Improving Context Dependency in Language Generation by Aligning with Historycited by 6
  7. 2023Tackling Hallucinations in Neural Chart Summarizationcited by 6
  8. 2024A Data-Centric Approach To Generate Faithful and High Quality Patient Summaries with Large Language Modelscited by 6
  9. 2021Towards objectively evaluating the quality of generated medical summariescited by 5
  10. 2022Evaluation of Question Answering Systems: Complexity of judging a natural languagecited by 5
  11. 2024The Pitfalls of Defining Hallucinationcited by 5
  12. 2025SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generationcited by 5
  13. 2025OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMscited by 4
  14. 2025Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Modelscited by 4
  15. 2023Context-Aware Document Simplificationcited by 3
  16. 2024Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTocited by 3
  17. 2026NLG Evaluation: Past, Present, Futurecited by 2
  18. 2022Comparing informativeness of an NLG chatbot vs graphical app in diet-information domaincited by 1
  19. 2024Question Generation in Knowledge-Driven Dialog: Explainability and Evaluationcited by 1
  20. 2026Precision Is Not Faithfulness: Coverage-Aware Evaluation of Grounded Generation with a Complete Oraclecited by 1
  21. 2025CCNU at SemEval-2025 Task 3: Leveraging Internal and External Knowledge of Large Language Models for Multilingual Hallucination Annotationcited by 0
  22. 2026Pitwall: Faithful Natural-Language Race-Strategy Briefings from a Calibrated Real-Time Monte Carlo Enginecited by 0