mlsanityData to 7 Oct 2026

Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation

arXiv 2104.07555 ↗First posted 15 Apr 2021cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

QuestEval is a reference-less metric used in text-to-text tasks, that compares the generated summaries directly to the source text, by automatically asking and answering questions. Its adaptation to Data-to-Text tasks is not straightforward, as it requires multimodal Question Generation and Answering systems on the considered tasks, which are seldom available. To this purpose, we propose a method to build synthetic multimodal corpora enabling to train multimodal components for a data-QuestEval metric. The resulting metric is reference-less and multimodal; it obtains state-of-the-art correlations with human judgment on the WebNLG and WikiBio benchmarks. We make data-QuestEval's code and models available for reproducibility purpose, as part of the QuestEval project.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 24 of its 39 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

24 of 39 references.

References · 24

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  6. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  7. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  8. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  9. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  10. 2021QuestEval: Summarization Asks for Fact-based Evaluationcited by 241
  11. 2017Why We Need New Evaluation Metrics for NLGcited by 232
  12. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
12 more
  1. 2021FeTaQA: Free-form Table Question Answeringcited by 174
  2. 2019Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasetscited by 173
  3. 2019Handling Divergent Reference Texts when Evaluating Table-to-Text Generationcited by 119
  4. 2020Text-to-Text Pre-Training for Data-to-Text Taskscited by 104
  5. 2019Answers Unite! Unsupervised Metrics for Reinforced Summarization Modelscited by 92
  6. 2019Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challengecited by 86
  7. 2019Hierarchical Graph Network for Multi-hop Question Answeringcited by 83
  8. 2021Controlling Hallucinations at Word Level in Data-to-Text Generationcited by 41
  9. 2020Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMRcited by 25
  10. 2017A Semantic QA-Based Approach for Text Summarization Evaluationcited by 17
  11. 2019Automatic Quality Estimation for Natural Language Generation: Ranting (Jointly Rating and Ranking)cited by 4
  12. 2020Reference and Document Aware Semantic Evaluation Methods for Korean Language Summarizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 5; 2023: 3; 2024: 2; 2025: 6; 2026: 2.

21 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
  3. 2021A Comprehensive Assessment of Dialog Evaluation Metricscited by 90
  4. 2021CONFIT: Toward Faithful Dialogue Summarization with Linguistically-Informed Contrastive Fine-tuningcited by 44
  5. 2024Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Modelscited by 44
  6. 2021MixQG: Neural Question Generation with Mixed Answer Typescited by 28
  7. 2022EAGER: Asking and Answering Questions for Automatic Reward Shaping in Language-guided RLcited by 23
  8. 2022RQUGE: Reference-Free Metric for Evaluating Question Generation by Answering the Questioncited by 17
  9. 2023On Evaluating the Integration of Reasoning and Action in LLM Agents with Database Question Answeringcited by 10
  10. 2025Reference-free Evaluation Metrics for Text Generation: A Surveycited by 9
  11. 2023Sequence-Level Certainty Reduces Hallucination In Knowledge-Grounded Dialogue Generationcited by 5
  12. 2025SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generationcited by 5
9 more
  1. 2022MaskEval: Weighted MLM-Based Evaluation for Text Summarization and Simplificationcited by 3
  2. 2025SteLLA: A Structured Grading System Using LLMs with RAGcited by 3
  3. 2024Ontology-Free General-Domain Knowledge Graph-to-Text Generation Dataset Synthesis using Large Language Modelcited by 2
  4. 2025TabXEval: Why this is a Bad Table? An eXhaustive Rubric for Table Evaluationcited by 2
  5. 2023Joint Representations of Text and Knowledge Graphs for Retrieval and Evaluationcited by 1
  6. 2025JUDGEBERT: Assessing Legal Meaning Preservation Between Sentencescited by 0
  7. 2025TabReX : Tabular Referenceless eXplainable Evaluationcited by 0
  8. 2026A Systematic Evaluation of Positional Bias in Multi-Video Summarization with MLLMscited by 0
  9. 2026XQDT: eXplainable and Quantitative Data-Text Alignment Metric with Feedback Signalscited by 0