# Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs

arXiv: https://arxiv.org/abs/2207.00551 · HTML: https://mlsanity.com/paper/2207.00551/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 45 references that are in the map.
- First posted on arXiv 1 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015)
- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives](https://mlsanity.com/paper/1802.07623/) (2018)
- [Algorithmic Recourse: from Counterfactual Explanations to Interventions](https://mlsanity.com/paper/2002.06278/) (2020)
- [Learning Model-Agnostic Counterfactual Explanations for Tabular Data](https://mlsanity.com/paper/1910.09398/) (2019)
- [Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches](https://mlsanity.com/paper/1912.05100/) (2019)
- [On quantitative aspects of model interpretability](https://mlsanity.com/paper/2007.07584/) (2020)
- [Knowledge Tracing with Sequential Key-Value Memory Networks](https://mlsanity.com/paper/1910.13197/) (2019)
- [AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling](https://mlsanity.com/paper/1807.00154/) (2018)
- [Towards Interpretable Deep Learning Models for Knowledge Tracing](https://mlsanity.com/paper/2005.06139/) (2020)

## Built on by (most cited first, 17 of 17)

- [MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks](https://mlsanity.com/paper/2309.14118/) (2023)
- [Towards Auditing Large Language Models: Improving Text-based Stereotype Detection](https://mlsanity.com/paper/2311.14126/) (2023)
- [The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations](https://mlsanity.com/paper/2307.00364/) (2023)
- [Intrinsic User-Centric Interpretability through Global Mixture of Experts](https://mlsanity.com/paper/2402.02933/) (2024)
- [RIPPLE: Concept-Based Interpretation for Raw Time Series Models in Education](https://mlsanity.com/paper/2212.01133/) (2022)
- [Trusting the Explainers: Teacher Validation of Explainable Artificial Intelligence for Course Design](https://mlsanity.com/paper/2212.08955/) (2022)
- [Automated Assessment of Encouragement and Warmth in Classrooms Leveraging Multimodal Emotional Features and ChatGPT](https://mlsanity.com/paper/2404.15310/) (2024)
- [Why do explanations fail? A typology and discussion on failures in XAI](https://mlsanity.com/paper/2405.13474/) (2024)
- [Using Think-Aloud Data to Understand Relations between Self-Regulation Cycle Characteristics and Student Performance in Intelligent Tutoring Systems](https://mlsanity.com/paper/2312.05675/) (2023)
- [Stereotype Detection in LLMs: A Multiclass, Explainable, and Benchmark-Driven Approach](https://mlsanity.com/paper/2404.01768/) (2024)
- [Deep Learning for Educational Data Science](https://mlsanity.com/paper/2404.19675/) (2024)
- [Evaluating Algorithmic Bias in Models for Predicting Academic Performance of Filipino Students](https://mlsanity.com/paper/2405.09821/) (2024)
- [Exploring the Relationship Between Feature Attribution Methods and Model Performance](https://mlsanity.com/paper/2405.13957/) (2024)
- [Interpret3C: Interpretable Student Clustering Through Individualized Feature Selection](https://mlsanity.com/paper/2407.11979/) (2024)
- [Do Not Trust a Model Because It is Confident: Uncovering and Characterizing Unknown Unknowns to Student Success Predictors in Online-Based Learning](https://mlsanity.com/paper/2212.08532/) (2022)
- [Multimodal Analytics of Cybersecurity Crisis Preparation Exercises: What Predicts Success?](https://mlsanity.com/paper/2603.28553/) (2026)
- [Stable and Faithful Explanations for Knowledge Tracing](https://mlsanity.com/paper/2609.28502/) (2026)

## Abstract (arXiv)

Neural networks are ubiquitous in applied machine learning for education. Their pervasive success in predictive performance comes alongside a severe weakness, the lack of explainability of their decisions, especially relevant in human-centric fields. We implement five state-of-the-art methodologies for explaining black-box machine learning models (LIME, PermutationSHAP, KernelSHAP, DiCE, CEM) and examine the strengths of each approach on the downstream task of student performance prediction for five massive open online courses. Our experiments demonstrate that the families of explainers do not agree with each other on feature importance for the same Bidirectional LSTM models with the same representative set of students. We use Principal Component Analysis, Jensen-Shannon distance, and Spearman's rank-order correlation to quantitatively cross-examine explanations across methods and courses. Furthermore, we validate explainer performance across curriculum-based prerequisite relationships. Our results come to the concerning conclusion that the choice of explainer is an important decision and is in fact paramount to the interpretation of the predictive results, even more so than the course the model is trained on. Source code and models are released at http://github.com/epfl-ml4ed/evaluating-explainers.