# Synthetic Student Responses: LLM-Extracted Features for IRT Difficulty Parameter Estimation

arXiv: https://arxiv.org/abs/2602.00034 · HTML: https://mlsanity.com/paper/2602.00034/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 30 references that are in the map.
- First posted on arXiv 18 Jan 2026.

## Builds on (most cited first, 5 of 5)

- [Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solving](https://mlsanity.com/paper/2405.12205/) (2024)
- [Improving Large Language Model Fine-tuning for Solving Math Problems](https://mlsanity.com/paper/2310.10047/) (2023)
- [R2DE: a NLP approach to estimating IRT parameters of newly generated questions](https://mlsanity.com/paper/2001.07569/) (2020)
- [Introducing a framework to assess newly created questions with Natural Language Processing](https://mlsanity.com/paper/2004.13530/) (2020)
- [Prediction of Item Difficulty for Reading Comprehension Items by Creation of Annotated Item Repository](https://mlsanity.com/paper/2502.20663/) (2025)

## Built on by (most cited first, 1 of 1)

- [Aligning LLM-Simulated and Human Examinees for Psychometric Calibration: A Cognitive Diagnostic Profiling Approach](https://mlsanity.com/paper/2607.26317/) (2026)

## Abstract (arXiv)

Educational assessment relies heavily on knowing question difficulty, traditionally determined through resource-intensive pre-testing with students. This creates significant barriers for both classroom teachers and assessment developers. We investigate whether Item Response Theory (IRT) difficulty parameters can be accurately estimated without student testing by modeling the response process and explore the relative contribution of different feature types to prediction accuracy. Our approach combines traditional linguistic features with pedagogical insights extracted using Large Language Models (LLMs), including solution step count, cognitive complexity, and potential misconceptions. We implement a two-stage process: first training a neural network to predict how students would respond to questions, then deriving difficulty parameters from these simulated response patterns. Using a dataset of over 250,000 student responses to mathematics questions, our model achieves a Pearson correlation of approximately 0.78 between predicted and actual difficulty parameters on completely unseen questions.