# Results and Insights from Diagnostic Questions: The NeurIPS 2020 Education Challenge

arXiv: https://arxiv.org/abs/2104.04034 · HTML: https://mlsanity.com/paper/2104.04034/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2021.

## Built on by (most cited first, 18 of 18)

- [EduAgent: Generative Student Agents in Learning](https://mlsanity.com/paper/2404.07963/) (2024)
- [Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation](https://mlsanity.com/paper/2402.01512/) (2024)
- [Simultaneous Missing Value Imputation and Structure Learning with Groups](https://mlsanity.com/paper/2110.08223/) (2021)
- [DGEKT: A Dual Graph Ensemble Learning Method for Knowledge Tracing](https://mlsanity.com/paper/2211.12881/) (2022)
- [DiVERT: Distractor Generation with Variational Errors Represented as Text for Math Multiple-choice Questions](https://mlsanity.com/paper/2406.19356/) (2024)
- [Learnersourcing in the Age of AI: Student, Educator and Machine Partnerships for Content Creation](https://mlsanity.com/paper/2306.06386/) (2023)
- [DeepQR: Neural-based Quality Ratings for Learnersourced Multiple-Choice Questions](https://mlsanity.com/paper/2111.10058/) (2021)
- [A Review of Data Mining in Personalized Education: Current Trends and Future Prospects](https://mlsanity.com/paper/2402.17236/) (2024)
- [Tailoring Diagnostic Modeling to Individual Learners: Personalized Distractor Generation via MCTS-Guided Reasoning Reconstruction](https://mlsanity.com/paper/2508.11184/) (2025)
- [Assessing the Performance of Online Students -- New Data, New Approaches, Improved Accuracy](https://mlsanity.com/paper/2109.01753/) (2021)
- [Not All Options Are Created Equal: Textual Option Weighting for Token-Efficient LLM-Based Knowledge Tracing](https://mlsanity.com/paper/2410.12872/) (2024)
- [Model-Free Assessment of Simulator Fidelity via Quantile Curves](https://mlsanity.com/paper/2512.05024/) (2025)
- [Introducing instance label correlation in multiple instance learning. Application to cancer detection on histopathological images](https://mlsanity.com/paper/2310.19359/) (2023)
- [Student Answer Forecasting: Transformer-Driven Answer Choice Prediction for Language Learning](https://mlsanity.com/paper/2405.20079/) (2024)
- [STRIVE: A Think & Improve Approach with Iterative Refinement for Enhancing Question Quality Estimation](https://mlsanity.com/paper/2504.05693/) (2025)
- [Recovering Stranded Discrimination in Knowledge Tracing: Per-Item Bias Correction via Empirical-Bayes Shrinkage](https://mlsanity.com/paper/2606.14123/) (2026)
- [Simulating Disengaged Students to Evaluate LLM-based Tutors](https://mlsanity.com/paper/2609.12331/) (2026)
- [An Information-Theoretic Evaluation Framework for Benchmark and Model Diagnosis in Knowledge Tracing](https://mlsanity.com/paper/2610.06988/) (2026)

## Abstract (arXiv)

This competition concerns educational diagnostic questions, which are pedagogically effective, multiple-choice questions (MCQs) whose distractors embody misconceptions. With a large and ever-increasing number of such questions, it becomes overwhelming for teachers to know which questions are the best ones to use for their students. We thus seek to answer the following question: how can we use data on hundreds of millions of answers to MCQs to drive automatic personalized learning in large-scale learning scenarios where manual personalization is infeasible? Success in using MCQ data at scale helps build more intelligent, personalized learning platforms that ultimately improve the quality of education en masse. To this end, we introduce a new, large-scale, real-world dataset and formulate 4 data mining tasks on MCQs that mimic real learning scenarios and target various aspects of the above question in a competition setting at NeurIPS 2020. We report on our NeurIPS competition in which nearly 400 teams submitted approximately 4000 submissions, with encouragingly diverse and effective approaches to each of our tasks.