# Better Distractions: Transformer-based Distractor Generation and Multiple Choice Question Filtering

arXiv: https://arxiv.org/abs/2010.09598 · HTML: https://mlsanity.com/paper/2010.09598/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 17 of 17)

- [A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education](https://mlsanity.com/paper/2312.03173/) (2023)
- [Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation](https://mlsanity.com/paper/2402.01512/) (2024)
- [BERT-based distractor generation for Swedish reading comprehension questions using a small-scale dataset](https://mlsanity.com/paper/2108.03973/) (2021)
- [Is Your Large Language Model Knowledgeable or a Choices-Only Cheater?](https://mlsanity.com/paper/2407.01992/) (2024)
- [HumanVBench: Probing Human-Centric Video Understanding in MLLMs with Automatically Synthesized Benchmarks](https://mlsanity.com/paper/2412.17574/) (2024)
- [Leaf: Multiple-Choice Question Generation](https://mlsanity.com/paper/2201.09012/) (2022)
- [Generating Plausible Distractors for Multiple-Choice Questions via Student Choice Prediction](https://mlsanity.com/paper/2501.13125/) (2025)
- [DisGeM: Distractor Generation for Multiple Choice Questions with Span Masking](https://mlsanity.com/paper/2409.18263/) (2024)
- [The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical Challenges](https://mlsanity.com/paper/2406.18900/) (2024)
- [DISTO: Evaluating Textual Distractors for Multi-Choice Questions using Negative Sampling based Approach](https://mlsanity.com/paper/2304.04881/) (2023)
- [D-GEN: Automatic Distractor Generation and Evaluation for Reliable Assessment of Generative Model](https://mlsanity.com/paper/2504.13439/) (2025)
- [Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design](https://mlsanity.com/paper/2603.12826/) (2026)
- [Auditing MCQA Benchmarks through Probability Landscapes](https://mlsanity.com/paper/2608.30372/) (2026)
- [EMBRACE: Evaluation and Modifications for Boosting RACE](https://mlsanity.com/paper/2305.08433/) (2023)
- [Unsupervised multiple choices question answering via universal corpus](https://mlsanity.com/paper/2402.17333/) (2024)
- [DGRC: An Effective Fine-tuning Framework for Distractor Generation in Chinese Multi-choice Reading Comprehension](https://mlsanity.com/paper/2405.19139/) (2024)
- [Corpus Prevalence of Multiple-Choice Question Options](https://mlsanity.com/paper/2602.17377/) (2026)

## Abstract (arXiv)

For the field of education, being able to generate semantically correct and educationally relevant multiple choice questions (MCQs) could have a large impact. While question generation itself is an active research topic, generating distractors (the incorrect multiple choice options) receives much less attention. A missed opportunity, since there is still a lot of room for improvement in this area. In this work, we train a GPT-2 language model to generate three distractors for a given question and text context, using the RACE dataset. Next, we train a BERT language model to answer MCQs, and use this model as a filter, to select only questions that can be answered and therefore presumably make sense. To evaluate our work, we start by using text generation metrics, which show that our model outperforms earlier work on distractor generation (DG) and achieves state-of-the-art performance. Also, by calculating the question answering ability, we show that larger base models lead to better performance. Moreover, we conducted a human evaluation study, which confirmed the quality of the generated questions, but showed no statistically significant effect of the QA filter.