# A Gold Standard Methodology for Evaluating Accuracy in Data-To-Text Systems

arXiv: https://arxiv.org/abs/2011.03992 · HTML: https://mlsanity.com/paper/2011.03992/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2020.

## Built on by (most cited first, 30 of 34)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text](https://mlsanity.com/paper/2202.06935/) (2022)
- [VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models](https://mlsanity.com/paper/2406.16338/) (2024)
- [A New Benchmark and Reverse Validation Method for Passage-level Hallucination Detection](https://mlsanity.com/paper/2310.06498/) (2023)
- [Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation](https://mlsanity.com/paper/2110.05456/) (2021)
- [Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices](https://mlsanity.com/paper/2408.09169/) (2024)
- [Underreporting of errors in NLG output, and what to do about it](https://mlsanity.com/paper/2108.01182/) (2021)
- [Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment](https://mlsanity.com/paper/2310.08372/) (2023)
- [Data-to-text Generation with Variational Sequential Planning](https://mlsanity.com/paper/2202.13756/) (2022)
- [Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation](https://mlsanity.com/paper/2401.10186/) (2024)
- [Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis](https://mlsanity.com/paper/2309.05217/) (2023)
- [Face4RAG: Factual Consistency Evaluation for Retrieval Augmented Generation in Chinese](https://mlsanity.com/paper/2407.01080/) (2024)
- [Generation Challenges: Results of the Accuracy Evaluation Shared Task](https://mlsanity.com/paper/2108.05644/) (2021)
- [Generating Full Length Wikipedia Biographies: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographies](https://mlsanity.com/paper/2204.05879/) (2022)
- [LLMs as Span Annotators: A Comparative Study of LLMs and Humans](https://mlsanity.com/paper/2504.08697/) (2025)
- [EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models](https://mlsanity.com/paper/2505.11405/) (2025)
- [HistAlign: Improving Context Dependency in Language Generation by Aligning with History](https://mlsanity.com/paper/2305.04782/) (2023)
- [Tackling Hallucinations in Neural Chart Summarization](https://mlsanity.com/paper/2308.00399/) (2023)
- [A Data-Centric Approach To Generate Faithful and High Quality Patient Summaries with Large Language Models](https://mlsanity.com/paper/2402.15422/) (2024)
- [Towards objectively evaluating the quality of generated medical summaries](https://mlsanity.com/paper/2104.04412/) (2021)
- [Evaluation of Question Answering Systems: Complexity of judging a natural language](https://mlsanity.com/paper/2209.12617/) (2022)
- [The Pitfalls of Defining Hallucination](https://mlsanity.com/paper/2401.07897/) (2024)
- [SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation](https://mlsanity.com/paper/2502.13674/) (2025)
- [OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMs](https://mlsanity.com/paper/2503.11858/) (2025)
- [Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models](https://mlsanity.com/paper/2504.19061/) (2025)
- [Context-Aware Document Simplification](https://mlsanity.com/paper/2305.06274/) (2023)
- [Improving Factual Accuracy of Neural Table-to-Text Output by Addressing Input Problems in ToTTo](https://mlsanity.com/paper/2404.04103/) (2024)
- [NLG Evaluation: Past, Present, Future](https://mlsanity.com/paper/2605.23715/) (2026)
- [Comparing informativeness of an NLG chatbot vs graphical app in diet-information domain](https://mlsanity.com/paper/2206.13435/) (2022)

## Abstract (arXiv)

Most Natural Language Generation systems need to produce accurate texts. We propose a methodology for high-quality human evaluation of the accuracy of generated texts, which is intended to serve as a gold-standard for accuracy evaluations of data-to-text systems. We use our methodology to evaluate the accuracy of computer generated basketball summaries. We then show how our gold standard evaluation can be used to validate automated metrics