# QUDEVAL: The Evaluation of Questions Under Discussion Discourse Parsing

arXiv: https://arxiv.org/abs/2310.14520 · HTML: https://mlsanity.com/paper/2310.14520/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 55 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 19 of 19)

- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Evaluation of Text Generation: A Survey](https://mlsanity.com/paper/2006.14799/) (2020)
- [Towards a Better Metric for Evaluating Question Generation Systems](https://mlsanity.com/paper/1808.10192/) (2018)
- [Conditional Generation with a Question-Answering Blueprint](https://mlsanity.com/paper/2207.00397/) (2022)
- [Answer-based Adversarial Training for Generating Clarification Questions](https://mlsanity.com/paper/1904.02281/) (2019)
- [Controllable Open-ended Question Generation with A New Question Type Ontology](https://mlsanity.com/paper/2107.00152/) (2021)
- [Inquisitive Question Generation for High Level Text Comprehension](https://mlsanity.com/paper/2010.01657/) (2020)
- [Generating Question-Answer Hierarchies](https://mlsanity.com/paper/1906.02622/) (2019)
- [ChainCQG: Flow-Aware Conversational Question Generation](https://mlsanity.com/paper/2102.02864/) (2021)
- [Discourse Comprehension: A Question Answering Framework to Represent Sentence Connections](https://mlsanity.com/paper/2111.00701/) (2021)
- [Generating Information-Seeking Conversations from Unlabeled Documents](https://mlsanity.com/paper/2205.12609/) (2022)
- [Elaborative Simplification as Implicit Questions Under Discussion](https://mlsanity.com/paper/2305.10387/) (2023)
- [Discourse Analysis via Questions and Answers: Parsing Dependency Structures of Questions Under Discussion](https://mlsanity.com/paper/2210.05905/) (2022)
- [Using Natural Language Explanations to Rescale Human Judgments](https://mlsanity.com/paper/2305.14770/) (2023)
- [CoHS-CQG: Context and History Selection for Conversational Question Generation](https://mlsanity.com/paper/2209.06652/) (2022)
- [CONSISTENT: Open-Ended Question Generation From News Articles](https://mlsanity.com/paper/2210.11536/) (2022)
- [End-to-end Neural Information Status Classification](https://mlsanity.com/paper/2109.02753/) (2021)

## Built on by (most cited first, 19 of 19)

- [ResearchQA: Evaluating Scholarly Question Answering at Scale Across 75 Fields with Survey-Mined Questions and Rubrics](https://mlsanity.com/paper/2509.00496/) (2025)
- [Conformal Linguistic Calibration: Trading-off between Factuality and Specificity](https://mlsanity.com/paper/2502.19110/) (2025)
- [InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification](https://mlsanity.com/paper/2401.16475/) (2024)
- [QUDsim: Quantifying Discourse Similarities in LLM-Generated Text](https://mlsanity.com/paper/2504.09373/) (2025)
- [Behavioral Analysis of Information Salience in Large Language Models](https://mlsanity.com/paper/2502.14613/) (2025)
- [Fine-Grained Modeling of Narrative Context: A Coherence Perspective via Retrospective Questions](https://mlsanity.com/paper/2402.13551/) (2024)
- [NLI under the Microscope: What Atomic Hypothesis Decomposition Reveals](https://mlsanity.com/paper/2502.08080/) (2025)
- [What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentations](https://mlsanity.com/paper/2502.08279/) (2025)
- [Which questions should I answer? Salience Prediction of Inquisitive Questions](https://mlsanity.com/paper/2404.10917/) (2024)
- [Pretraining on the Test Set Is No Longer All You Need: A Debate-Driven Approach to QA Benchmarks](https://mlsanity.com/paper/2507.17747/) (2025)
- [QUDSELECT: Selective Decoding for Questions Under Discussion Parsing](https://mlsanity.com/paper/2408.01046/) (2024)
- [Explanatory Summarization with Discourse-Driven Planning](https://mlsanity.com/paper/2504.19339/) (2025)
- [Discursive Circuits: How Do Language Models Understand Discourse Relations?](https://mlsanity.com/paper/2510.11210/) (2025)
- [A Survey of QUD Models for Discourse Processing](https://mlsanity.com/paper/2502.15573/) (2025)
- [How to Engage Your Readers? Generating Guiding Questions to Promote Active Reading](https://mlsanity.com/paper/2407.14309/) (2024)
- [Which course? Discourse! Teaching Discourse and Generation in the Era of LLMs](https://mlsanity.com/paper/2602.02878/) (2026)
- [DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering](https://mlsanity.com/paper/2604.15140/) (2026)
- [Multimodal QUD: Inquisitive Questions from Scientific Figures](https://mlsanity.com/paper/2604.23733/) (2026)
- [Generating Edit-Inducing Questions for AI Research Manuscripts](https://mlsanity.com/paper/2609.36617/) (2026)

## Abstract (arXiv)

Questions Under Discussion (QUD) is a versatile linguistic framework in which discourse progresses as continuously asking questions and answering them. Automatic parsing of a discourse to produce a QUD structure thus entails a complex question generation task: given a document and an answer sentence, generate a question that satisfies linguistic constraints of QUD and can be grounded in an anchor sentence in prior context. These questions are known to be curiosity-driven and open-ended. This work introduces the first framework for the automatic evaluation of QUD parsing, instantiating the theoretical constraints of QUD in a concrete protocol. We present QUDeval, a dataset of fine-grained evaluation of 2,190 QUD questions generated from both fine-tuned systems and LLMs. Using QUDeval, we show that satisfying all constraints of QUD is still challenging for modern LLMs, and that existing evaluation metrics poorly approximate parser quality. Encouragingly, human-authored QUDs are scored highly by our human evaluators, suggesting that there is headroom for further progress on language modeling to improve both QUD parsing and QUD evaluation.