# Exploring Automated Distractor Generation for Math Multiple-choice Questions via Large Language Models

arXiv: https://arxiv.org/abs/2404.02124 · HTML: https://mlsanity.com/paper/2404.02124/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 39 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations](https://mlsanity.com/paper/2212.09865/) (2022)
- [Math Word Problem Generation with Mathematical Consistency and Problem Context Constraints](https://mlsanity.com/paper/2109.04546/) (2021)
- [Automatic Distractor Generation for Multiple Choice Questions in Standard Tests](https://mlsanity.com/paper/2011.13100/) (2020)
- [Generating Distractors for Reading Comprehension Questions from Real Examinations](https://mlsanity.com/paper/1809.02768/) (2018)
- [Automatic Short Math Answer Grading via In-context Meta-learning](https://mlsanity.com/paper/2205.15219/) (2022)
- [Interpretable Math Word Problem Solution Generation Via Step-by-step Planning](https://mlsanity.com/paper/2306.00784/) (2023)
- [BERT-based distractor generation for Swedish reading comprehension questions using a small-scale dataset](https://mlsanity.com/paper/2108.03973/) (2021)
- [Math Operation Embeddings for Open-ended Solution Analysis and Feedback](https://mlsanity.com/paper/2104.12047/) (2021)

## Built on by (most cited first, 15 of 15)

- [MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research](https://mlsanity.com/paper/2503.13399/) (2025)
- [Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errors](https://mlsanity.com/paper/2510.11502/) (2025)
- [HumanVBench: Probing Human-Centric Video Understanding in MLLMs with Automatically Synthesized Benchmarks](https://mlsanity.com/paper/2412.17574/) (2024)
- [Math Multiple Choice Question Generation via Human-Large Language Model Collaboration](https://mlsanity.com/paper/2405.00864/) (2024)
- [The StudyChat Dataset: Analyzing Student Dialogues With ChatGPT in an Artificial Intelligence Course](https://mlsanity.com/paper/2503.07928/) (2025)
- [Do LLMs Make Mistakes Like Students? Exploring Natural Alignment between Language Models and Human Error Patterns](https://mlsanity.com/paper/2502.15140/) (2025)
- [From Text to Visuals: Using LLMs to Generate Math Diagrams with Vector Graphics](https://mlsanity.com/paper/2503.07429/) (2025)
- [FoundationalASSIST: An Educational Dataset for Foundational Knowledge Tracing and Pedagogical Grounding of LLMs](https://mlsanity.com/paper/2602.00070/) (2026)
- [KCIF: Knowledge-Conditioned Instruction Following](https://mlsanity.com/paper/2410.12972/) (2024)
- [EvalYaks: Instruction Tuning Datasets and LoRA Fine-tuned Models for Automated Scoring of CEFR B2 Speaking Assessment Transcripts](https://mlsanity.com/paper/2408.12226/) (2024)
- [Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design](https://mlsanity.com/paper/2603.12826/) (2026)
- [A self-evolving multi-role collaborative framework with fine-grained difficulty guidance for innovative mathematical problem generation](https://mlsanity.com/paper/2601.11792/) (2026)
- [Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation using Knowledge Graphs and Large Language Models](https://mlsanity.com/paper/2604.10748/) (2026)
- [MCQ Difficulty Prediction via Modeling Learner Heterogeneity Using Data-Driven Cognitive Profiling](https://mlsanity.com/paper/2605.16290/) (2026)
- [The Architect, the Adversary, and the Judge: Closed-Loop Generation of Standards-Aligned Assessment Items at Scale](https://mlsanity.com/paper/2609.26087/) (2026)

## Abstract (arXiv)

Multiple-choice questions (MCQs) are ubiquitous in almost all levels of education since they are easy to administer, grade, and are a reliable format in assessments and practices. One of the most important aspects of MCQs is the distractors, i.e., incorrect options that are designed to target common errors or misconceptions among real students. To date, the task of crafting high-quality distractors largely remains a labor and time-intensive process for teachers and learning content designers, which has limited scalability. In this work, we study the task of automated distractor generation in the domain of math MCQs and explore a wide variety of large language model (LLM)-based approaches, from in-context learning to fine-tuning. We conduct extensive experiments using a real-world math MCQ dataset and find that although LLMs can generate some mathematically valid distractors, they are less adept at anticipating common errors or misconceptions among real students.