# Evaluating Semantic Accuracy of Data-to-Text Generation with Natural Language Inference

arXiv: https://arxiv.org/abs/2011.10819 · HTML: https://mlsanity.com/paper/2011.10819/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2020.

## Built on by (most cited first, 30 of 43)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Factuality Enhanced Language Models for Open-Ended Text Generation](https://mlsanity.com/paper/2206.04624/) (2022)
- [LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models](https://mlsanity.com/paper/2307.07889/) (2023)
- [Conditional Generation with a Question-Answering Blueprint](https://mlsanity.com/paper/2207.00397/) (2022)
- [Towards Knowledge-Grounded Counter Narrative Generation for Hate Speech](https://mlsanity.com/paper/2106.11783/) (2021)
- [AI Hallucinations: A Misnomer Worth Clarifying](https://mlsanity.com/paper/2401.06796/) (2024)
- [Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation](https://mlsanity.com/paper/2110.05456/) (2021)
- [Investigating Reasons for Disagreement in Natural Language Inference](https://mlsanity.com/paper/2209.03392/) (2022)
- [How Far Can We Extract Diverse Perspectives from Large Language Models?](https://mlsanity.com/paper/2311.09799/) (2023)
- [Neural Pipeline for Zero-Shot Data-to-Text Generation](https://mlsanity.com/paper/2203.16279/) (2022)
- [Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment](https://mlsanity.com/paper/2310.08372/) (2023)
- [MENLI: Robust Evaluation Metrics from Natural Language Inference](https://mlsanity.com/paper/2208.07316/) (2022)
- [Understanding Counterspeech for Online Harm Mitigation](https://mlsanity.com/paper/2307.04761/) (2023)
- [Evaluating Differentially Private Synthetic Data Generation in High-Stakes Domains](https://mlsanity.com/paper/2410.08327/) (2024)
- [AgreeSum: Agreement-Oriented Multi-Document Summarization](https://mlsanity.com/paper/2106.02278/) (2021)
- [Innovations in Neural Data-to-text Generation: A Survey](https://mlsanity.com/paper/2207.12571/) (2022)
- [Generating Full Length Wikipedia Biographies: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographies](https://mlsanity.com/paper/2204.05879/) (2022)
- [Critic-Driven Decoding for Mitigating Hallucinations in Data-to-text Generation](https://mlsanity.com/paper/2310.16964/) (2023)
- [A Survey of Automatic Hallucination Evaluation on Natural Language Generation](https://mlsanity.com/paper/2404.12041/) (2024)
- [The Effects of Hallucinations in Synthetic Training Data for Relation Extraction](https://mlsanity.com/paper/2410.08393/) (2024)
- [Mind the Labels: Describing Relations in Knowledge Graphs With Pretrained Models](https://mlsanity.com/paper/2210.07373/) (2022)
- [Tackling Hallucinations in Neural Chart Summarization](https://mlsanity.com/paper/2308.00399/) (2023)
- [Towards objectively evaluating the quality of generated medical summaries](https://mlsanity.com/paper/2104.04412/) (2021)
- [The Pitfalls of Defining Hallucination](https://mlsanity.com/paper/2401.07897/) (2024)
- [SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation](https://mlsanity.com/paper/2502.13674/) (2025)
- [Towards a Zero-Data, Controllable, Adaptive Dialog System](https://mlsanity.com/paper/2403.17582/) (2024)
- [PAGED: A Benchmark for Procedural Graphs Extraction from Documents](https://mlsanity.com/paper/2408.03630/) (2024)
- [OpineSum: Entailment-based self-training for abstractive opinion summarization](https://mlsanity.com/paper/2212.10791/) (2022)
- [ASPIRO: Any-shot Structured Parsing-error-Induced ReprOmpting for Consistent Data-to-Text Generation](https://mlsanity.com/paper/2310.17877/) (2023)
- [NLG Evaluation: Past, Present, Future](https://mlsanity.com/paper/2605.23715/) (2026)

## Abstract (arXiv)

A major challenge in evaluating data-to-text (D2T) generation is measuring the semantic accuracy of the generated text, i.e. checking if the output text contains all and only facts supported by the input data. We propose a new metric for evaluating the semantic accuracy of D2T generation based on a neural model pretrained for natural language inference (NLI). We use the NLI model to check textual entailment between the input data and the output text in both directions, allowing us to reveal omissions or hallucinations. Input data are converted to text for NLI using trivial templates. Our experiments on two recent D2T datasets show that our metric can achieve high accuracy in identifying erroneous system outputs.