# Closed-book Question Generation via Contrastive Learning

arXiv: https://arxiv.org/abs/2210.06781 · HTML: https://mlsanity.com/paper/2210.06781/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 68 references that are in the map.
- First posted on arXiv 13 Oct 2022.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [How Much Knowledge Can You Pack Into the Parameters of a Language Model?](https://mlsanity.com/paper/2002.08910/) (2020)
- [The NarrativeQA Reading Comprehension Challenge](https://mlsanity.com/paper/1712.07040/) (2017)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://mlsanity.com/paper/2006.03659/) (2020)
- [PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://mlsanity.com/paper/2102.07033/) (2021)
- [Learning to Ask: Neural Question Generation for Reading Comprehension](https://mlsanity.com/paper/1705.00106/) (2017)
- [Synthetic QA Corpora Generation with Roundtrip Consistency](https://mlsanity.com/paper/1906.05416/) (2019)
- [SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization](https://mlsanity.com/paper/2106.01890/) (2021)
- [Contrastive Learning for Many-to-many Multilingual Neural Machine Translation](https://mlsanity.com/paper/2105.09501/) (2021)
- [Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering](https://mlsanity.com/paper/1909.06356/) (2019)
- [Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA?](https://mlsanity.com/paper/2106.01561/) (2021)
- [Unsupervised Question Answering by Cloze Translation](https://mlsanity.com/paper/1906.04980/) (2019)
- [GooAQ: Open Question Answering with Diverse Answer Types](https://mlsanity.com/paper/2104.08727/) (2021)
- [Answer-based Adversarial Training for Generating Clarification Questions](https://mlsanity.com/paper/1904.02281/) (2019)
- [Visual Question Generation as Dual Task of Visual Question Answering](https://mlsanity.com/paper/1709.07192/) (2017)
- [Learning to Ask Appropriate Questions in Conversational Recommendation](https://mlsanity.com/paper/2105.04774/) (2021)
- [Data Augmentation with Hierarchical SQL-to-Question Generation for Cross-domain Text-to-SQL Parsing](https://mlsanity.com/paper/2103.02227/) (2021)
- [Improving Neural Question Generation using Answer Separation](https://mlsanity.com/paper/1809.02393/) (2018)
- [Semantic Parsing with Dual Learning](https://mlsanity.com/paper/1907.05343/) (2019)

## Built on by (most cited first, 6 of 6)

- [Co2PT: Mitigating Bias in Pre-trained Language Models through Counterfactual Contrastive Prompt Tuning](https://mlsanity.com/paper/2310.12490/) (2023)
- [HiPrompt: Few-Shot Biomedical Knowledge Fusion via Hierarchy-Oriented Prompting](https://mlsanity.com/paper/2304.05973/) (2023)
- [MuG: A Multimodal Classification Benchmark on Game Data with Tabular, Textual, and Visual Fields](https://mlsanity.com/paper/2302.02978/) (2023)
- [Towards Diverse and Effective Question-Answer Pair Generation from Children Storybooks](https://mlsanity.com/paper/2306.06605/) (2023)
- [Named Entity Recognition via Machine Reading Comprehension: A Multi-Task Learning Approach](https://mlsanity.com/paper/2309.11027/) (2023)
- [Unsupervised Candidate Answer Extraction through Differentiable Masker-Reconstructor Model](https://mlsanity.com/paper/2310.13106/) (2023)

## Abstract (arXiv)

Question Generation (QG) is a fundamental NLP task for many downstream applications. Recent studies on open-book QG, where supportive answer-context pairs are provided to models, have achieved promising progress. However, generating natural questions under a more practical closed-book setting that lacks these supporting documents still remains a challenge. In this work, we propose a new QG model for this closed-book setting that is designed to better understand the semantics of long-form abstractive answers and store more information in its parameters through contrastive learning and an answer reconstruction module. Through experiments, we validate the proposed QG model on both public datasets and a new WikiCQA dataset. Empirical results show that the proposed QG model outperforms baselines in both automatic evaluation and human evaluation. In addition, we show how to leverage the proposed model to improve existing question-answering systems. These results further indicate the effectiveness of our QG model for enhancing closed-book question-answering tasks.