mlsanityData to 14 Oct 2026

Handling Divergent Reference Texts when Evaluating Table-to-Text Generation

arXiv 1906.01081 ↗First posted 3 Jun 2019cs.CLcited by 119

Not on the map. Why?

Abstract (arXiv)

Automatically constructed datasets for generating text from semi-structured data (tables), such as WikiBio, often contain reference texts that diverge from the information in the corresponding semi-structured data. We show that metrics which rely solely on the reference texts, such as BLEU and ROUGE, show poor correlation with human judgments when those references diverge. We propose a new metric, PARENT, which aligns n-grams from the reference and generated texts to the semi-structured data before computing their precision and recall. Through a large scale human evaluation study of table-to-text models for WikiBio, we show that PARENT correlates with human judgments better than existing text generation metrics. We also adapt and evaluate the information extraction based evaluation proposed by Wiseman et al (2017), and show that PARENT has comparable correlation to it, while being easier to use. We show that PARENT is also applicable when the reference texts are elicited from humans using the data from the WebNLG challenge.

Key facts. Cited by 119 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 21; 2022: 25; 2023: 27; 2024: 29; 2025: 8; 2026: 9.

119 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  2. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  3. 2024Large Language Models: A Surveycited by 525
  4. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  5. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 341
  6. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  7. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  8. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  9. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 188
  10. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 155
  11. 2021A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generationcited by 138
  12. 2021HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generationcited by 118
38 more
  1. 2021Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Groundingcited by 106
  2. 2024Best Practices and Lessons Learned on Synthetic Datacited by 106
  3. 2021Evaluating Attribution in Dialogue Systems: The BEGIN Benchmarkcited by 77
  4. 2021Planning with Learned Entity Prompts for Abstractive Summarizationcited by 75
  5. 2022Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generationcited by 75
  6. 2023Exposing Attention Glitches with Flip-Flop Language Modelingcited by 75
  7. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  8. 2021Plan-then-Generate: Controlled Data-to-Text Generation via Planningcited by 52
  9. 2021Perturbation CheckLists for Evaluating NLG Evaluation Metricscited by 50
  10. 2024ALOHa: A New Measure for Hallucination in Captioning Modelscited by 45
  11. 2024Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Modelscited by 44
  12. 2021Controlling Hallucinations at Word Level in Data-to-Text Generationcited by 41
  13. 2023Language Models Hallucinate, but May Excel at Fact Verificationcited by 37
  14. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 37
  15. 2022Input-Tuning: Adapting Unfamiliar Inputs to Frozen Pretrained Modelscited by 34
  16. 2022Robust (Controlled) Table-to-Text Generation with Structure-Aware Equivariance Learningcited by 32
  17. 2021Focus Attention: Promoting Faithfulness and Diversity in Summarizationcited by 30
  18. 2023FactCHD: Benchmarking Fact-Conflicting Hallucination Detectioncited by 29
  19. 2022A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generationcited by 28
  20. 2022Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methodscited by 27
  21. 2022Towards Improving Faithfulness in Abstractive Summarizationcited by 27
  22. 2021Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric Viewcited by 24
  23. 2021Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluationcited by 21
  24. 2021Sketch and Refine: Towards Faithful and Informative Table-to-Text Generationcited by 21
  25. 2022CoNT: Contrastive Neural Text Generationcited by 21
  26. 2024Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practicescited by 20
  27. 2024Tree-of-Table: Unleashing the Power of LLMs for Enhanced Large-Scale Table Understandingcited by 20
  28. 2021SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasetscited by 18
  29. 2022ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Modelscited by 14
  30. 2024PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamicscited by 14
  31. 2024Unifying Structured Data as Graph for Data-to-Text Pre-Trainingcited by 13
  32. 2024Fakes of Varying Shades: How Warning Affects Human Perception and Engagement Regarding LLM Hallucinationscited by 12
  33. 2021Stage-wise Fine-tuning for Graph-to-Text Generationcited by 11
  34. 2021Truth-Conditional Captioning of Time Series Datacited by 11
  35. 2022Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisonscited by 10
  36. 2022Innovations in Neural Data-to-text Generation: A Surveycited by 10
  37. 2022TaTa: A Multilingual Table-to-Text Dataset for African Languagescited by 10
  38. 2022Time-aware Prompting for Text Generationcited by 10

50 most cited of 119 All 119 (JSON)