Recovering Stranded Discrimination in Knowledge Tracing: Per-Item Bias Correction via Empirical-Bayes Shrinkage
Not on the map. Why?
Abstract (arXiv)
Deployed knowledge-tracing models are typically frozen after training, yet systematic per-item logit bias arises, from limited per-item expressivity in backbone architectures and from post-deployment shifts in item properties, degrading prediction quality. Global post-hoc calibrators such as Platt scaling, temperature scaling, and isotonic regression improve probability estimates but leave discriminative ability, as measured by AUC, unchanged. This AUC invariance is a structural consequence of monotone score-only transforms; recovering the stranded discrimination requires conditioning on item identity. We propose SLC (State-space Logit Correction), which converts binary observations to Gaussian pseudo-observations via Laplace/IRLS, applies empirical-Bayes shrinkage through a Kalman smoother, and fits an offset-Platt link. The state-space formulation also yields a detectability bound that characterizes the Bernoulli information floor, explaining why temporal tracking provides no benefit at current data densities. Across four datasets, five backbones, and three seeds, SLC improves AUC on all four datasets and NLL on three, with the advantage concentrating on sparse items. Cross-domain controls suggest that the same phenomenon can arise beyond education when the deployed backbone leaves entity-level bias.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 36 references that are in the map. First posted on arXiv 12 Jun 2026.
Builds on
12 of 36 references.
References · 12
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2017Neural Collaborative Filteringcited by 1,111
- 2020Long-tail learning via logit adjustmentcited by 618
- 2015Deep Knowledge Tracingcited by 332
- 2020Context-Aware Attentive Knowledge Tracingcited by 174
- 2019A Self-Attentive model for Knowledge Tracingcited by 172
- 2022Better Uncertainty Calibration via Proper Scores for Classification and Beyondcited by 65
- 2021Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibrationcited by 48
- 2019Field-aware Calibration: A Simple and Empirically Strong Method for Reliable Probabilistic Predictionscited by 19
- 2021Results and Insights from Diagnostic Questions: The NeurIPS 2020 Education Challengecited by 18
- 2023Beyond In-Domain Scenarios: Robust Density-Aware Calibrationcited by 14
- 2023Variational Temporal IRT: Fast, Accurate, and Explainable Inference of Dynamic Learner Proficiencycited by 3
Built on by
Not cited yet.