# Do Not Trust a Model Because It is Confident: Uncovering and Characterizing Unknown Unknowns to Student Success Predictors in Online-Based Learning

arXiv: https://arxiv.org/abs/2212.08532 · HTML: https://mlsanity.com/paper/2212.08532/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 34 references that are in the map.
- First posted on arXiv 16 Dec 2022.

## Builds on (most cited first, 7 of 7)

- [Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making](https://mlsanity.com/paper/2001.02114/) (2020)
- [How model accuracy and explanation fidelity influence user trust](https://mlsanity.com/paper/1907.12652/) (2019)
- [Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs](https://mlsanity.com/paper/2207.00551/) (2022)
- [Exploratory Machine Learning with Unknown Unknowns](https://mlsanity.com/paper/2002.01605/) (2020)
- [Meta Transfer Learning for Early Success Prediction in MOOCs](https://mlsanity.com/paper/2205.01064/) (2022)
- [Student Success Prediction in MOOCs](https://mlsanity.com/paper/1711.06349/) (2017)
- [Experts' View on Challenges and Needs for Fairness in Artificial Intelligence for Education](https://mlsanity.com/paper/2207.01490/) (2022)

## Abstract (arXiv)

Student success models might be prone to develop weak spots, i.e., examples hard to accurately classify due to insufficient representation during model creation. This weakness is one of the main factors undermining users' trust, since model predictions could for instance lead an instructor to not intervene on a student in need. In this paper, we unveil the need of detecting and characterizing unknown unknowns in student success prediction in order to better understand when models may fail. Unknown unknowns include the students for which the model is highly confident in its predictions, but is actually wrong. Therefore, we cannot solely rely on the model's confidence when evaluating the predictions quality. We first introduce a framework for the identification and characterization of unknown unknowns. We then assess its informativeness on log data collected from flipped courses and online courses using quantitative analyses and interviews with instructors. Our results show that unknown unknowns are a critical issue in this domain and that our framework can be applied to support their detection. The source code is available at https://github.com/epfl-ml4ed/unknown-unknowns.