# BERT-based distractor generation for Swedish reading comprehension questions using a small-scale dataset

arXiv: https://arxiv.org/abs/2108.03973 · HTML: https://mlsanity.com/paper/2108.03973/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 47 references that are in the map.
- First posted on arXiv 9 Aug 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Crowdsourcing Multiple Choice Science Questions](https://mlsanity.com/paper/1707.06209/) (2017)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection](https://mlsanity.com/paper/2004.10643/) (2020)
- [BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model](https://mlsanity.com/paper/1902.04094/) (2019)
- [Playing with Words at the National Library of Sweden -- Making a Swedish BERT](https://mlsanity.com/paper/2007.01658/) (2020)
- [Probabilistically Masked Language Model Capable of Autoregressive Generation in Arbitrary Word Order](https://mlsanity.com/paper/2004.11579/) (2020)
- [Automatic Distractor Generation for Multiple Choice Questions in Standard Tests](https://mlsanity.com/paper/2011.13100/) (2020)
- [A BERT-based Distractor Generation Scheme with Multi-tasking and Negative Answer Training Strategies](https://mlsanity.com/paper/2010.05384/) (2020)
- [Better Distractions: Transformer-based Distractor Generation and Multiple Choice Question Filtering](https://mlsanity.com/paper/2010.09598/) (2020)
- [Co-Attention Hierarchical Network: Generating Coherent Long Distractors for Reading Comprehension](https://mlsanity.com/paper/1911.08648/) (2019)
- [Quinductor: a multilingual data-driven method for generating reading-comprehension questions using Universal Dependencies](https://mlsanity.com/paper/2103.10121/) (2021)

## Built on by (most cited first, 12 of 12)

- [A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education](https://mlsanity.com/paper/2312.03173/) (2023)
- [Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation](https://mlsanity.com/paper/2402.01512/) (2024)
- [Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning](https://mlsanity.com/paper/2308.03234/) (2023)
- [Distractor generation for multiple-choice questions with predictive prompting and large language models](https://mlsanity.com/paper/2307.16338/) (2023)
- [Exploring Automated Distractor Generation for Math Multiple-choice Questions via Large Language Models](https://mlsanity.com/paper/2404.02124/) (2024)
- [DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions](https://mlsanity.com/paper/2406.19356/) (2024)
- [SweCTRL-Mini: a data-transparent Transformer-based large language model for controllable text generation in Swedish](https://mlsanity.com/paper/2304.13994/) (2023)
- [DISTO: Evaluating Textual Distractors for Multi-Choice Questions using Negative Sampling based Approach](https://mlsanity.com/paper/2304.04881/) (2023)
- [Auditing MCQA Benchmarks through Probability Landscapes](https://mlsanity.com/paper/2608.30372/) (2026)
- [Automatically generating question-answer pairs for assessing basic reading comprehension in Swedish](https://mlsanity.com/paper/2211.15568/) (2022)
- [EMBRACE: Evaluation and Modifications for Boosting RACE](https://mlsanity.com/paper/2305.08433/) (2023)
- [DGRC: An Effective Fine-tuning Framework for Distractor Generation in Chinese Multi-choice Reading Comprehension](https://mlsanity.com/paper/2405.19139/) (2024)

## Abstract (arXiv)

An important part when constructing multiple-choice questions (MCQs) for reading comprehension assessment are the distractors, the incorrect but preferably plausible answer options. In this paper, we present a new BERT-based method for automatically generating distractors using only a small-scale dataset. We also release a new such dataset of Swedish MCQs (used for training the model), and propose a methodology for assessing the generated distractors. Evaluation shows that from a student's perspective, our method generated one or more plausible distractors for more than 50% of the MCQs in our test set. From a teacher's perspective, about 50% of the generated distractors were deemed appropriate. We also do a thorough analysis of the results.