# QURIOUS: Question Generation Pretraining for Text Generation

arXiv: https://arxiv.org/abs/2004.11026 · HTML: https://mlsanity.com/paper/2004.11026/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2020.

## Built on by (most cited first, 12 of 12)

- [Planning with Learned Entity Prompts for Abstractive Summarization](https://mlsanity.com/paper/2104.07606/) (2021)
- [All You May Need for VQA are Image Captions](https://mlsanity.com/paper/2205.01883/) (2022)
- [Quiz-Style Question Generation for News Stories](https://mlsanity.com/paper/2102.09094/) (2021)
- [Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling](https://mlsanity.com/paper/2402.17019/) (2024)
- [Improving Unsupervised Question Answering via Summarization-Informed Question Generation](https://mlsanity.com/paper/2109.07954/) (2021)
- [Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask](https://mlsanity.com/paper/2204.02908/) (2022)
- [Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization](https://mlsanity.com/paper/2212.10449/) (2022)
- [QAScore -- An Unsupervised Unreferenced Metric for the Question Generation Evaluation](https://mlsanity.com/paper/2210.04320/) (2022)
- [AQUALLM: Audio Question Answering Data Generation Using Large Language Models](https://mlsanity.com/paper/2312.17343/) (2023)
- [Automated Reading Passage Generation with OpenAI's Large Language Model](https://mlsanity.com/paper/2304.04616/) (2023)
- [The Role of Summarization in Generative Agents: A Preliminary Perspective](https://mlsanity.com/paper/2305.01253/) (2023)
- [On the Evaluation of Answer-Agnostic Paragraph-level Multi-Question Generation](https://mlsanity.com/paper/2203.04464/) (2022)

## Abstract (arXiv)

Recent trends in natural language processing using pretraining have shifted focus towards pretraining and fine-tuning approaches for text generation. Often the focus has been on task-agnostic approaches that generalize the language modeling objective. We propose question generation as a pretraining method, which better aligns with the text generation objectives. Our text generation models pretrained with this method are better at understanding the essence of the input and are better language models for the target task. When evaluated on two text generation tasks, abstractive summarization and answer-focused question generation, our models result in state-of-the-art performances in terms of automatic metrics. Human evaluators also found our summaries and generated questions to be more natural, concise and informative.