# Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation

arXiv: https://arxiv.org/abs/2603.15547 · HTML: https://mlsanity.com/paper/2603.15547/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Mathematical reasoning, one of 17 directions in LLM reasoning.
- Builds on 9 of its 58 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Placed in

- LLM reasoning / [Mathematical reasoning](https://mlsanity.com/lr/mathematical-reasoning/)

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025)
- [Learning to Make MISTAKEs: Modeling Incorrect Student Thinking And Key Errors](https://mlsanity.com/paper/2510.11502/) (2025)
- [Novice Learner and Expert Tutor: Evaluating Math Reasoning Abilities of Large Language Models with Misconceptions](https://mlsanity.com/paper/2310.02439/) (2023)

## Built on by (most cited first, 1 of 1)

- [Taxonomy-Targeted Error Generation for Quantitative Reasoning](https://mlsanity.com/paper/2605.29007/) (2026)

## Abstract (arXiv)

Modeling student misconceptions in a realistic manner is critical for AI in education. In this work, we examine how large language models (LLMs) reason about misconceptions when generating distractor answers for multiple-choice questions (MCQs), a task that requires producing answers that are incorrect, yet plausible. We introduce a taxonomy over reasoning strategies for distractor generation that is grounded in learning-science literature and empirical observation, which we apply to LLM-generated reasoning traces across math and science MCQs. On the math dataset, we find that models follow a misconception-based process with potentially high diagnostic value: they recover the correct solution, articulate student errors, simulate them, and select plausible candidates. On the science dataset, on the other hand, they tend to follow a less robust approach based on semantic similarity to the correct answer. We find the most frequent failure modes to be that the model is unable to generate a correct solution or that it discards plausible distractor candidates when performing selection. Providing the correct solution in the prompt yields a relative improvement of 6.4% in alignment with human-authored distractors, highlighting the critical role of anchoring distractor generation to the correct solution. Together, our findings offer an interpretable view of how LLMs model incorrect student reasoning.