# Simplifying Paragraph-level Question Generation via Transformer Language Models

arXiv: https://arxiv.org/abs/2005.01107 · HTML: https://mlsanity.com/paper/2005.01107/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2020.

## Built on by (most cited first, 12 of 12)

- [ScanQA: 3D Question Answering for Spatial Scene Understanding](https://mlsanity.com/paper/2112.10482/) (2021)
- [Scalable Educational Question Generation with Pre-trained Language Models](https://mlsanity.com/paper/2305.07871/) (2023)
- [QASem Parsing: Text-to-text Modeling of QA-based Semantics](https://mlsanity.com/paper/2205.11413/) (2022)
- [A Novel Approach to Scalable and Automatic Topic-Controlled Question Generation in Education](https://mlsanity.com/paper/2501.05220/) (2025)
- [CoHS-CQG: Context and History Selection for Conversational Question Generation](https://mlsanity.com/paper/2209.06652/) (2022)
- [Evaluation of Question Answering Systems: Complexity of judging a natural language](https://mlsanity.com/paper/2209.12617/) (2022)
- [Automatic Generation and Evaluation of Reading Comprehension Test Items with Large Language Models](https://mlsanity.com/paper/2404.07720/) (2024)
- [Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trends](https://mlsanity.com/paper/2401.07518/) (2024)
- [Data Processing Matters: SRPH-Konvergen AI's Machine Translation System for WMT'21](https://mlsanity.com/paper/2111.10513/) (2021)
- [Reflecting in the Reflection: Integrating a Socratic Questioning Framework into Automated AI-Based Question Generation](https://mlsanity.com/paper/2601.14798/) (2026)
- [Releasing the CRaQAn (Coreference Resolution in Question-Answering): An open-source dataset and dataset creation methodology using instruction-following models](https://mlsanity.com/paper/2311.16338/) (2023)
- [DragonVerseQA: Open-Domain Long-Form Context-Aware Question-Answering](https://mlsanity.com/paper/2412.16694/) (2024)

## Abstract (arXiv)

Question generation (QG) is a natural language generation task where a model is trained to ask questions corresponding to some input text. Most recent approaches frame QG as a sequence-to-sequence problem and rely on additional features and mechanisms to increase performance; however, these often increase model complexity, and can rely on auxiliary data unavailable in practical use. A single Transformer-based unidirectional language model leveraging transfer learning can be used to produce high quality questions while disposing of additional task-specific complexity. Our QG model, finetuned from GPT-2 Small, outperforms several paragraph-level QG baselines on the SQuAD dataset by 0.95 METEOR points. Human evaluators rated questions as easy to answer, relevant to their context paragraph, and corresponding well to natural human speech. Also introduced is a new set of baseline scores on the RACE dataset, which has not previously been used for QG tasks. Further experimentation with varying model capacities and datasets with non-identification type questions is recommended in order to further verify the robustness of pretrained Transformer-based LMs as question generators.