# R2DE: a NLP approach to estimating IRT parameters of newly generated questions

arXiv: https://arxiv.org/abs/2001.07569 · HTML: https://mlsanity.com/paper/2001.07569/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2020.

## Built on by (most cited first, 15 of 15)

- [How to Select Datapoints for Efficient Human Evaluation of NLG Models?](https://mlsanity.com/paper/2501.18251/) (2025)
- [Reasoning and Sampling-Augmented MCQ Difficulty Prediction via LLMs](https://mlsanity.com/paper/2503.08551/) (2025)
- [A quantitative study of NLP approaches to question difficulty estimation](https://mlsanity.com/paper/2305.10236/) (2023)
- [Towards a Holistic Understanding of Mathematical Questions with Contrastive Pre-training](https://mlsanity.com/paper/2301.07558/) (2023)
- [Proactive and Reactive Engagement of Artificial Intelligence Methods for Education: A Review](https://mlsanity.com/paper/2301.10231/) (2023)
- [From Text to Parameters: Predicting Item Parameters from Embedding Regularization with Reliability and Design Ceilings](https://mlsanity.com/paper/2607.07141/) (2026)
- [Building a Question and Answer System for News Domain](https://mlsanity.com/paper/2105.05744/) (2021)
- [Synthetic Student Responses: LLM-Extracted Features for IRT Difficulty Parameter Estimation](https://mlsanity.com/paper/2602.00034/) (2026)
- [Estimating Item Difficulty with Large Language Models as Experts](https://mlsanity.com/paper/2605.18562/) (2026)
- [Transferable Student Performance Modeling for Intelligent Tutoring Systems](https://mlsanity.com/paper/2202.03980/) (2022)
- [Response-free item difficulty modelling for multiple-choice items with fine-tuned transformers: Component-wise representation and multi-task learning](https://mlsanity.com/paper/2605.16991/) (2026)
- [Structure-Aware Modeling of Multiple-Choice Questions Improves Automatic Difficulty Estimation](https://mlsanity.com/paper/2606.08988/) (2026)
- [Automated item evaluation: Predicting item acceptance and rejection using LLM-generated critiques](https://mlsanity.com/paper/2608.06609/) (2026)
- [Evaluation of pre-trained models for pedagogical assessment of novel AI-assisted educational questions](https://mlsanity.com/paper/2609.27749/) (2026)
- [What Makes Something Hard(er)? Explaining Question Difficulty in Natural Language](https://mlsanity.com/paper/2610.01627/) (2026)

## Abstract (arXiv)

The main objective of exams consists in performing an assessment of students' expertise on a specific subject. Such expertise, also referred to as skill or knowledge level, can then be leveraged in different ways (e.g., to assign a grade to the students, to understand whether a student might need some support, etc.). Similarly, the questions appearing in the exams have to be assessed in some way before being used to evaluate students. Standard approaches to questions' assessment are either subjective (e.g., assessment by human experts) or introduce a long delay in the process of question generation (e.g., pretesting with real students). In this work we introduce R2DE (which is a Regressor for Difficulty and Discrimination Estimation), a model capable of assessing newly generated multiple-choice questions by looking at the text of the question and the text of the possible choices. In particular, it can estimate the difficulty and the discrimination of each question, as they are defined in Item Response Theory. We also present the results of extensive experiments we carried out on a real world large scale dataset coming from an e-learning platform, showing that our model can be used to perform an initial assessment of newly created questions and ease some of the problems that arise in question generation.