# Assessing the Performance of Online Students -- New Data, New Approaches, Improved Accuracy

arXiv: https://arxiv.org/abs/2109.01753 · HTML: https://mlsanity.com/paper/2109.01753/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 74 references that are in the map.
- First posted on arXiv 4 Sep 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Context-Aware Attentive Knowledge Tracing](https://mlsanity.com/paper/2007.12324/) (2020)
- [A Self-Attentive model for Knowledge Tracing](https://mlsanity.com/paper/1907.06837/) (2019)
- [Dynamic Key-Value Memory Networks for Knowledge Tracing](https://mlsanity.com/paper/1611.08108/) (2016)
- [EKT: Exercise-aware Knowledge Tracing for Student Performance Prediction](https://mlsanity.com/paper/1906.05658/) (2019)
- [EdNet: A Large-Scale Hierarchical Dataset in Education](https://mlsanity.com/paper/1912.03072/) (2019)
- [Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing](https://mlsanity.com/paper/2002.07033/) (2020)
- [Instructions and Guide for Diagnostic Questions: The NeurIPS 2020 Education Challenge](https://mlsanity.com/paper/2007.12061/) (2020)
- [SAINT+: Integrating Temporal Features for EdNet Correctness Prediction](https://mlsanity.com/paper/2010.12042/) (2020)
- [Deep-IRT: Make Deep Learning Based Knowledge Tracing Explainable Using Item Response Theory](https://mlsanity.com/paper/1904.11738/) (2019)
- [GIKT: A Graph-based Interaction Model for Knowledge Tracing](https://mlsanity.com/paper/2009.05991/) (2020)
- [DAS3H: Modeling Student Learning and Forgetting for Optimally Scheduling Distributed Practice of Skills](https://mlsanity.com/paper/1905.06873/) (2019)
- [Results and Insights from Diagnostic Questions: The NeurIPS 2020 Education Challenge](https://mlsanity.com/paper/2104.04034/) (2021)
- [Logistic Knowledge Tracing: A Constrained Framework for Learner Modeling](https://mlsanity.com/paper/2005.00869/) (2020)
- [pyBKT: An Accessible Python Library of Bayesian Knowledge Tracing Models](https://mlsanity.com/paper/2105.00385/) (2021)
- [HGKT: Introducing Hierarchical Exercise Graph for Knowledge Tracing](https://mlsanity.com/paper/2006.16915/) (2020)
- [Modeling Knowledge Acquisition from Multiple Learning Resource Types](https://mlsanity.com/paper/2006.13390/) (2020)
- [Extending Deep Knowledge Tracing: Inferring Interpretable Knowledge and Predicting Post-System Performance](https://mlsanity.com/paper/1910.12597/) (2019)
- [LANA: Towards Personalized Deep Knowledge Tracing Through Distinguishable Interactive Sequences](https://mlsanity.com/paper/2105.06266/) (2021)
- [On the Interpretability of Deep Learning Based Models for Knowledge Tracing](https://mlsanity.com/paper/2101.11335/) (2021)
- [MUSE: Multi-Scale Temporal Features Evolution for Knowledge Tracing](https://mlsanity.com/paper/2102.00228/) (2021)

## Built on by (most cited first, 2 of 2)

- [A Toolbox for Modelling Engagement with Educational Videos](https://mlsanity.com/paper/2401.05424/) (2023)
- [Offline Policy Evaluation for Reinforcement Learning with Adaptively Collected Data](https://mlsanity.com/paper/2306.14063/) (2023)

## Abstract (arXiv)

We consider the problem of assessing the changing performance levels of individual students as they go through online courses. This student performance (SP) modeling problem is a critical step for building adaptive online teaching systems. Specifically, we conduct a study of how to utilize various types and large amounts of student log data to train accurate machine learning (ML) models that predict the performance of future students. This study is the first to use four very large sets of student data made available recently from four distinct intelligent tutoring systems. Our results include a new ML approach that defines a new state of the art for logistic regression based SP modeling, improving over earlier methods in several ways: First, we achieve improved accuracy by introducing new features that can be easily computed from conventional question-response logs (e.g., the pattern in the student 's most recent answers). Second, we take advantage of features of the student history that go beyond question-response pairs (e.g., features such as which video segments the student watched, or skipped) as well as information about prerequisite structure in the curriculum. Third, we train multiple specialized SP models for different aspects of the curriculum (e.g., specializing in early versus later segments of the student history), then combine these specialized models to create a group prediction of the SP. Taken together, these innovations yield an average AUC score across these four datasets of 0.808 compared to the previous best logistic regression approach score of 0.767, and also outperforming state-of-the-art deep neural net approaches. Importantly, we observe consistent improvements from each of our three methodological innovations, in each dataset, suggesting that our methods are of general utility and likely to produce improvements for other online tutoring systems as well.