# A Lightweight Method to Generate Unanswerable Questions in English

arXiv: https://arxiv.org/abs/2310.19403 · HTML: https://mlsanity.com/paper/2310.19403/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 30 Oct 2023.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Adversarial Examples for Evaluating Reading Comprehension Systems](https://mlsanity.com/paper/1707.07328/) (2017)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)
- [AllenNLP: A Deep Semantic Natural Language Processing Platform](https://mlsanity.com/paper/1803.07640/) (2018)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension](https://mlsanity.com/paper/1910.09753/) (2019)
- [Synthetic QA Corpora Generation with Roundtrip Consistency](https://mlsanity.com/paper/1906.05416/) (2019)
- [Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehension](https://mlsanity.com/paper/2002.00293/) (2020)
- [Simple and Effective Multi-Paragraph Reading Comprehension](https://mlsanity.com/paper/1710.10723/) (2017)
- [Semi-supervised sequence tagging with bidirectional language models](https://mlsanity.com/paper/1705.00108/) (2017)
- [Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation](https://mlsanity.com/paper/2104.08678/) (2021)
- [(QA)2: Question Answering with Questionable Assumptions](https://mlsanity.com/paper/2212.10003/) (2022)
- [Robust Machine Comprehension Models via Adversarial Training](https://mlsanity.com/paper/1804.06473/) (2018)
- [Making Neural QA as Simple as Possible but not Simpler](https://mlsanity.com/paper/1703.04816/) (2017)
- [Read + Verify: Machine Reading Comprehension with Unanswerable Questions](https://mlsanity.com/paper/1808.05759/) (2018)
- [Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space](https://mlsanity.com/paper/2010.01475/) (2020)
- [Quantifying and Avoiding Unfair Qualification Labour in Crowdsourcing](https://mlsanity.com/paper/2105.12762/) (2021)
- [What Makes Reading Comprehension Questions Difficult?](https://mlsanity.com/paper/2203.06342/) (2022)
- [Improving Machine Reading Comprehension via Adversarial Training](https://mlsanity.com/paper/1911.03614/) (2019)
- [To Answer or Not to Answer? Improving Machine Reading Comprehension Model with Span-based Contrastive Learning](https://mlsanity.com/paper/2208.01299/) (2022)

## Abstract (arXiv)

If a question cannot be answered with the available information, robust systems for question answering (QA) should know _not_ to answer. One way to build QA models that do this is with additional training data comprised of unanswerable questions, created either by employing annotators or through automated methods for unanswerable question generation. To show that the model complexity of existing automated approaches is not justified, we examine a simpler data augmentation method for unanswerable question generation in English: performing antonym and entity swaps on answerable questions. Compared to the prior state-of-the-art, data generated with our training-free and lightweight strategy results in better models (+1.6 F1 points on SQuAD 2.0 data with BERT-large), and has higher human-judged relatedness and readability. We quantify the raw benefits of our approach compared to no augmentation across multiple encoder models, using different amounts of generated data, and also on TydiQA-MinSpan data (+9.3 F1 points with BERT-large). Our results establish swaps as a simple but strong baseline for future work.