mlsanityData to 7 Oct 2026

Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference

arXiv 2011.10819 ↗First posted 21 Nov 2020cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

A major challenge in evaluating data-to-text (D2T) generation is measuring the semantic accuracy of the generated text, i.e. checking if the output text contains all and only facts supported by the input data. We propose a new metric for evaluating the semantic accuracy of D2T generation based on a neural model pretrained for natural language inference (NLI). We use the NLI model to check textual entailment between the input data and the output text in both directions, allowing us to reveal omissions or hallucinations. Input data are converted to text for NLI using trivial templates. Our experiments on two recent D2T datasets show that our metric can achieve high accuracy in identifying erroneous system outputs.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 11; 2023: 12; 2024: 9; 2025: 4; 2026: 3.

43 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  3. 2023LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Modelscited by 66
  4. 2022Conditional Generation with a Question-Answering Blueprintcited by 44
  5. 2021Towards Knowledge-Grounded Counter Narrative Generation for Hate Speechcited by 41
  6. 2024AI Hallucinations: A Misnomer Worth Clarifyingcited by 34
  7. 2021Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generationcited by 29
  8. 2022Investigating Reasons for Disagreement in Natural Language Inferencecited by 29
  9. 2023How Far Can We Extract Diverse Perspectives from Large Language Models?cited by 22
  10. 2022Neural Pipeline for Zero-Shot Data-to-Text Generationcited by 19
  11. 2023Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignmentcited by 19
  12. 2022MENLI: Robust Evaluation Metrics from Natural Language Inferencecited by 18
31 more
  1. 2023Understanding Counterspeech for Online Harm Mitigationcited by 17
  2. 2024Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domainscited by 12
  3. 2021AgreeSum: Agreement-Oriented Multi-Document Summarizationcited by 10
  4. 2022Innovations in Neural Data-to-text Generation: A Surveycited by 10
  5. 2022Generating Full Length Wikipedia Biographies: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographiescited by 7
  6. 2023Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generationcited by 7
  7. 2024A Survey of Automatic Hallucination Evaluation on Natural Language Generationcited by 7
  8. 2024The Effects of Hallucinations in Synthetic Training Data for Relation Extractioncited by 7
  9. 2022Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Modelscited by 6
  10. 2023Tackling Hallucinations in Neural Chart Summarizationcited by 6
  11. 2021Towards objectively evaluating the quality of generated medical summariescited by 5
  12. 2024The Pitfalls of Defining Hallucinationcited by 5
  13. 2025SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generationcited by 5
  14. 2024Towards a Zero-Data, Controllable, Adaptive Dialog Systemcited by 4
  15. 2024PAGED: A Benchmark for Procedural Graphs Extraction from Documentscited by 4
  16. 2022OpineSum: Entailment-based self-training for abstractive opinion summarizationcited by 3
  17. 2023ASPIRO: Any-shot Structured Parsing-error-Induced ReprOmpting for Consistent Data-to-Text Generationcited by 2
  18. 2026NLG Evaluation: Past, Present, Futurecited by 2
  19. 2026Multilingual Unlearning in LLMs: Transfer, Dynamics, and Reversibilitycited by 2
  20. 2022nBIIG: A Neural BI Insights Generation System for Table Reportingcited by 1
  21. 2023Schema-Guided Semantic Accuracy: Faithfulness in Task-Oriented Dialogue Response Generationcited by 1
  22. 2023Joint Representations of Text and Knowledge Graphs for Retrieval and Evaluationcited by 1
  23. 2023Neural models for Factual Inconsistency Classification with Explanationscited by 1
  24. 2023Knowledge Graph for NLG in the context of conversational agentscited by 1
  25. 2023Compositional Generalization for Data-to-Text Generationcited by 1
  26. 2024Beyond Keywords: A Context-based Hybrid Approach to Mining Ethical Concern-related App Reviewscited by 1
  27. 2025Measuring and Mitigating Hallucinations in Vision-Language Dataset Generation for Remote Sensingcited by 1
  28. 2024Projective Methods for Mitigating Gender Bias in Pre-trained Language Modelscited by 0
  29. 2025CCNU at SemEval-2025 Task 3: Leveraging Internal and External Knowledge of Large Language Models for Multilingual Hallucination Annotationcited by 0
  30. 2025Real-World Summarization: When Evaluation Reaches Its Limitscited by 0
  31. 2026XQDT: eXplainable and Quantitative Data-Text Alignment Metric with Feedback Signalscited by 0