# Downstream Transformer Generation of Question-Answer Pairs with Preprocessing and Postprocessing Pipelines

arXiv: https://arxiv.org/abs/2205.07387 · HTML: https://mlsanity.com/paper/2205.07387/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 53 references that are in the map.
- First posted on arXiv 15 May 2022.

## Builds on (most cited first, 30 of 31)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation](https://mlsanity.com/paper/1603.08023/) (2016)
- [ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training](https://mlsanity.com/paper/2001.04063/) (2020)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Simple BERT Models for Relation Extraction and Semantic Role Labeling](https://mlsanity.com/paper/1904.05255/) (2019)
- [Semi-supervised sequence tagging with bidirectional language models](https://mlsanity.com/paper/1705.00108/) (2017)
- [Neural Question Generation from Text: A Preliminary Study](https://mlsanity.com/paper/1704.01792/) (2017)
- [Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering](https://mlsanity.com/paper/1909.06356/) (2019)
- [Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation](https://mlsanity.com/paper/1908.04942/) (2019)
- [Towards a Better Metric for Evaluating Question Generation Systems](https://mlsanity.com/paper/1808.10192/) (2018)
- [Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples](https://mlsanity.com/paper/1805.06556/) (2018)
- [Improving Neural Question Generation using Answer Separation](https://mlsanity.com/paper/1809.02393/) (2018)
- [MixQG: Neural Question Generation with Mixed Answer Types](https://mlsanity.com/paper/2110.08175/) (2021)
- [Quiz-Style Question Generation for News Stories](https://mlsanity.com/paper/2102.09094/) (2021)
- [Multi-hop Question Generation with Graph Convolutional Network](https://mlsanity.com/paper/2010.09240/) (2020)
- [Question-type Driven Question Generation](https://mlsanity.com/paper/1909.00140/) (2019)
- [Neural Generation of Diverse Questions using Answer Focus, Contextual and Linguistic Features](https://mlsanity.com/paper/1809.02637/) (2018)
- [Question Generation by Transformers](https://mlsanity.com/paper/1909.05017/) (2019)

## Built on by (most cited first, 5 of 5)

- [Evaluating the Performance of Large Language Models on GAOKAO Benchmark](https://mlsanity.com/paper/2305.12474/) (2023)
- [Multilingual Relation Classification via Efficient and Effective Prompting](https://mlsanity.com/paper/2210.13838/) (2022)
- [Automatic Generation of Multiple-Choice Questions](https://mlsanity.com/paper/2303.14576/) (2023)
- [Tag-Set-Sequence Learning for Generating Question-Answer Pairs](https://mlsanity.com/paper/2210.11608/) (2022)
- [Constructing Cloze Questions Generatively](https://mlsanity.com/paper/2410.04266/) (2024)

## Abstract (arXiv)

We present a system called TP3 to perform a downstream task of transformers on generating question-answer pairs (QAPs) from a given article. TP3 first finetunes pretrained transformers on QAP datasets, then uses a preprocessing pipeline to select appropriate answers, feeds the relevant sentences and the answer to the finetuned transformer to generate candidate QAPs, and finally uses a postprocessing pipeline to filter inadequate QAPs. In particular, using pretrained T5 models as transformers and the SQuAD dataset as the finetruning dataset, we show that TP3 generates satisfactory number of QAPs with high qualities on the Gaokao-EN dataset.