# Quiz-Style Question Generation for News Stories

arXiv: https://arxiv.org/abs/2102.09094 · HTML: https://mlsanity.com/paper/2102.09094/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 18 Feb 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [Crowdsourcing Multiple Choice Science Questions](https://mlsanity.com/paper/1707.06209/) (2017)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training](https://mlsanity.com/paper/2002.12804/) (2020)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering](https://mlsanity.com/paper/1909.06356/) (2019)
- [Generating Natural Questions About an Image](https://mlsanity.com/paper/1603.06059/) (2016)
- [Corpora Generation for Grammatical Error Correction](https://mlsanity.com/paper/1904.05780/) (2019)
- [Harvesting Paragraph-Level Question-Answer Pairs from Wikipedia](https://mlsanity.com/paper/1805.05942/) (2018)
- [Generating Representative Headlines for News Stories](https://mlsanity.com/paper/2001.09386/) (2020)
- [ClarQ: A large-scale and diverse dataset for Clarification Question Generation](https://mlsanity.com/paper/2006.05986/) (2020)
- [Knowledge Questions from Knowledge Graphs](https://mlsanity.com/paper/1610.09935/) (2016)
- [Multi-Reference Training with Pseudo-References for Neural Translation and Text Generation](https://mlsanity.com/paper/1808.09564/) (2018)
- [Let's Ask Again: Refine Network for Automatic Question Generation](https://mlsanity.com/paper/1909.05355/) (2019)
- [QURIOUS: Question Generation Pretraining for Text Generation](https://mlsanity.com/paper/2004.11026/) (2020)

## Built on by (most cited first, 23 of 23)

- [QuALITY: Question Answering with Long Input Texts, Yes!](https://mlsanity.com/paper/2112.08608/) (2021)
- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers](https://mlsanity.com/paper/2407.09413/) (2024)
- [ArchivalQA: A Large-scale Benchmark Dataset for Open Domain Question Answering over Historical News Collections](https://mlsanity.com/paper/2109.03438/) (2021)
- [MixQG: Neural Question Generation with Mixed Answer Types](https://mlsanity.com/paper/2110.08175/) (2021)
- [Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation](https://mlsanity.com/paper/2402.01512/) (2024)
- [Learning to Ask Like a Physician](https://mlsanity.com/paper/2206.02696/) (2022)
- [ChroniclingAmericaQA: A Large-scale Question Answering Dataset based on Historical American Newspaper Pages](https://mlsanity.com/paper/2403.17859/) (2024)
- [CoHS-CQG: Context and History Selection for Conversational Question Generation](https://mlsanity.com/paper/2209.06652/) (2022)
- [Is your benchmark truly adversarial? AdvScore: Evaluating Human-Grounded Adversarialness](https://mlsanity.com/paper/2406.16342/) (2024)
- [Closed-book Question Generation via Contrastive Learning](https://mlsanity.com/paper/2210.06781/) (2022)
- [ComplexTempQA:A 100m Dataset for Complex Temporal Question Answering](https://mlsanity.com/paper/2406.04866/) (2024)
- [DocTabQA: Answering Questions from Long Documents Using Tables](https://mlsanity.com/paper/2408.11490/) (2024)
- [Downstream Transformer Generation of Question-Answer Pairs with Preprocessing and Postprocessing Pipelines](https://mlsanity.com/paper/2205.07387/) (2022)
- [Generating Answer Candidates for Quizzes and Answer-Aware Question Generators](https://mlsanity.com/paper/2108.12898/) (2021)
- [Unified Question Generation with Continual Lifelong Learning](https://mlsanity.com/paper/2201.09696/) (2022)
- [PREME: Preference-based Meeting Exploration through an Interactive Questionnaire](https://mlsanity.com/paper/2205.02370/) (2022)
- [Automatic Generation of Multiple-Choice Questions](https://mlsanity.com/paper/2303.14576/) (2023)
- [DocVideoQA: Towards Comprehensive Understanding of Document-Centric Videos through Question Answering](https://mlsanity.com/paper/2503.15887/) (2025)
- [UniPoll: A Unified Social Media Poll Generation Framework via Multi-Objective Optimization](https://mlsanity.com/paper/2306.06851/) (2023)
- [How the Advent of Ubiquitous Large Language Models both Stymie and Turbocharge Dynamic Adversarial Question Generation](https://mlsanity.com/paper/2401.11185/) (2024)
- [Adversarial and Safely Scaled Question Generation](https://mlsanity.com/paper/2210.09467/) (2022)
- [EMBRACE: Evaluation and Modifications for Boosting RACE](https://mlsanity.com/paper/2305.08433/) (2023)

## Abstract (arXiv)

A large majority of American adults get at least some of their news from the Internet. Even though many online news products have the goal of informing their users about the news, they lack scalable and reliable tools for measuring how well they are achieving this goal, and therefore have to resort to noisy proxy metrics (e.g., click-through rates or reading time) to track their performance. As a first step towards measuring news informedness at a scale, we study the problem of quiz-style multiple-choice question generation, which may be used to survey users about their knowledge of recent news. In particular, we formulate the problem as two sequence-to-sequence tasks: question-answer generation (QAG) and distractor, or incorrect answer, generation (DG). We introduce NewsQuizQA, the first dataset intended for quiz-style question-answer generation, containing 20K human written question-answer pairs from 5K news article summaries. Using this dataset, we propose a series of novel techniques for applying large pre-trained Transformer encoder-decoder models, namely PEGASUS and T5, to the tasks of question-answer generation and distractor generation. We show that our models outperform strong baselines using both automated metrics and human raters. We provide a case study of running weekly quizzes on real-world users via the Google Surveys platform over the course of two months. We found that users generally found the automatically generated questions to be educational and enjoyable. Finally, to serve the research community, we are releasing the NewsQuizQA dataset.