# Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation

arXiv: https://arxiv.org/abs/2104.07555 · HTML: https://mlsanity.com/paper/2104.07555/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 24 of 24)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data](https://mlsanity.com/paper/2004.07347/) (2020)
- [QuestEval: Summarization Asks for Fact-based Evaluation](https://mlsanity.com/paper/2103.12693/) (2021)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets](https://mlsanity.com/paper/1908.07898/) (2019)
- [FeTaQA: Free-form Table Question Answering](https://mlsanity.com/paper/2104.00369/) (2021)
- [Handling Divergent Reference Texts when Evaluating Table-to-Text Generation](https://mlsanity.com/paper/1906.01081/) (2019)
- [Text-to-Text Pre-Training for Data-to-Text Tasks](https://mlsanity.com/paper/2005.10433/) (2020)
- [Answers Unite! Unsupervised Metrics for Reinforced Summarization Models](https://mlsanity.com/paper/1909.01610/) (2019)
- [Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge](https://mlsanity.com/paper/1901.07931/) (2019)
- [Hierarchical Graph Network for Multi-hop Question Answering](https://mlsanity.com/paper/1911.03631/) (2019)
- [Controlling Hallucinations at Word Level in Data-to-Text Generation](https://mlsanity.com/paper/2102.02810/) (2021)
- [Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR](https://mlsanity.com/paper/2008.08896/) (2020)
- [A Semantic QA-Based Approach for Text Summarization Evaluation](https://mlsanity.com/paper/1704.06259/) (2017)
- [Automatic Quality Estimation for Natural Language Generation: Ranting (Jointly Rating and Ranking)](https://mlsanity.com/paper/1910.04731/) (2019)
- [Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarization](https://mlsanity.com/paper/2005.03510/) (2020)

## Built on by (most cited first, 21 of 21)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text](https://mlsanity.com/paper/2202.06935/) (2022)
- [A Comprehensive Assessment of Dialog Evaluation Metrics](https://mlsanity.com/paper/2106.03706/) (2021)
- [CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuning](https://mlsanity.com/paper/2112.08713/) (2021)
- [Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models](https://mlsanity.com/paper/2407.04121/) (2024)
- [MixQG: Neural Question Generation with Mixed Answer Types](https://mlsanity.com/paper/2110.08175/) (2021)
- [EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RL](https://mlsanity.com/paper/2206.09674/) (2022)
- [RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Question](https://mlsanity.com/paper/2211.01482/) (2022)
- [On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answering](https://mlsanity.com/paper/2311.09721/) (2023)
- [Reference-free Evaluation Metrics for Text Generation: A Survey](https://mlsanity.com/paper/2501.12011/) (2025)
- [Sequence-Level Certainty Reduces Hallucination In Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2310.18794/) (2023)
- [SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation](https://mlsanity.com/paper/2502.13674/) (2025)
- [MaskEval: Weighted MLM-Based Evaluation for Text Summarization and Simplification](https://mlsanity.com/paper/2205.12394/) (2022)
- [Ontology-Free General-Domain Knowledge Graph-to-Text Generation Dataset Synthesis using Large Language Model](https://mlsanity.com/paper/2409.07088/) (2024)
- [SteLLA: A Structured Grading System Using LLMs with RAG](https://mlsanity.com/paper/2501.09092/) (2025)
- [TabXEval: Why this is a Bad Table? An eXhaustive Rubric for Table Evaluation](https://mlsanity.com/paper/2505.22176/) (2025)
- [Joint Representations of Text and Knowledge Graphs for Retrieval and Evaluation](https://mlsanity.com/paper/2302.14785/) (2023)
- [JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences](https://mlsanity.com/paper/2508.16870/) (2025)
- [TabReX : Tabular Referenceless eXplainable Evaluation](https://mlsanity.com/paper/2512.15907/) (2025)
- [A Systematic Evaluation of Positional Bias in Multi-Video Summarization with MLLMs](https://mlsanity.com/paper/2606.04596/) (2026)
- [XQDT: eXplainable and Quantitative Data-Text Alignment Metric with Feedback Signals](https://mlsanity.com/paper/2608.29948/) (2026)

## Abstract (arXiv)

QuestEval is a reference-less metric used in text-to-text tasks, that compares the generated summaries directly to the source text, by automatically asking and answering questions. Its adaptation to Data-to-Text tasks is not straightforward, as it requires multimodal Question Generation and Answering systems on the considered tasks, which are seldom available. To this purpose, we propose a method to build synthetic multimodal corpora enabling to train multimodal components for a data-QuestEval metric. The resulting metric is reference-less and multimodal; it obtains state-of-the-art correlations with human judgment on the WebNLG and WikiBio benchmarks. We make data-QuestEval's code and models available for reproducibility purpose, as part of the QuestEval project.