# On the Evaluation of Answer-Agnostic Paragraph-level Multi-Question Generation

arXiv: https://arxiv.org/abs/2203.04464 · HTML: https://mlsanity.com/paper/2203.04464/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 60 references that are in the map.
- First posted on arXiv 9 Mar 2022.

## Builds on (most cited first, 30 of 30)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training](https://mlsanity.com/paper/2001.04063/) (2020)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Training Question Answering Models From Synthetic Data](https://mlsanity.com/paper/2002.09599/) (2020)
- [Neural Question Generation from Text: A Preliminary Study](https://mlsanity.com/paper/1704.01792/) (2017)
- [End-to-End Synthetic Data Generation for Domain Adaptation of Question Answering Systems](https://mlsanity.com/paper/2010.06028/) (2020)
- [Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation](https://mlsanity.com/paper/1706.09799/) (2017)
- [Towards a Better Metric for Evaluating Question Generation Systems](https://mlsanity.com/paper/1808.10192/) (2018)
- [One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases](https://mlsanity.com/paper/1810.05241/) (2018)
- [Question Answering and Question Generation as Dual Tasks](https://mlsanity.com/paper/1706.02027/) (2017)
- [Machine Comprehension by Text-to-Text Neural Question Generation](https://mlsanity.com/paper/1705.02012/) (2017)
- [Learning to Ask Questions in Open-domain Conversational Systems with Typed Decoders](https://mlsanity.com/paper/1805.04843/) (2018)
- [Memory-Efficient Adaptive Optimization](https://mlsanity.com/paper/1901.11150/) (2019)
- [Question-type Driven Question Generation](https://mlsanity.com/paper/1909.00140/) (2019)
- [What's The Latest? A Question-driven News Chatbot](https://mlsanity.com/paper/2105.05392/) (2021)
- [A Joint Model for Question Answering and Question Generation](https://mlsanity.com/paper/1706.01450/) (2017)
- [QURIOUS: Question Generation Pretraining for Text Generation](https://mlsanity.com/paper/2004.11026/) (2020)
- [Neural Models for Key Phrase Detection and Question Generation](https://mlsanity.com/paper/1706.04560/) (2017)
- [A Unified Query-based Generative Model for Question Generation and Question Answering](https://mlsanity.com/paper/1709.01058/) (2017)
- [A Question Type Driven and Copy Loss Enhanced Frameworkfor Answer-Agnostic Neural Question Generation](https://mlsanity.com/paper/2005.11665/) (2020)
- [Evaluating for Diversity in Question Generation over Text](https://mlsanity.com/paper/2008.07291/) (2020)

## Built on by (most cited first, 1 of 1)

- [Multi-VQG: Generating Engaging Questions for Multiple Images](https://mlsanity.com/paper/2211.07441/) (2022)

## Abstract (arXiv)

We study the task of predicting a set of salient questions from a given paragraph without any prior knowledge of the precise answer. We make two main contributions. First, we propose a new method to evaluate a set of predicted questions against the set of references by using the Hungarian algorithm to assign predicted questions to references before scoring the assigned pairs. We show that our proposed evaluation strategy has better theoretical and practical properties compared to prior methods because it can properly account for the coverage of references. Second, we compare different strategies to utilize a pre-trained seq2seq model to generate and select a set of questions related to a given paragraph. The code is available.