# Trusting the Explainers: Teacher Validation of Explainable Artificial Intelligence for Course Design

arXiv: https://arxiv.org/abs/2212.08955 · HTML: https://mlsanity.com/paper/2212.08955/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 49 references that are in the map.
- First posted on arXiv 17 Dec 2022.

## Builds on (most cited first, 7 of 7)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives](https://mlsanity.com/paper/1802.07623/) (2018)
- [AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling](https://mlsanity.com/paper/1807.00154/) (2018)
- [Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs](https://mlsanity.com/paper/2207.00551/) (2022)
- [Meta Transfer Learning for Early Success Prediction in MOOCs](https://mlsanity.com/paper/2205.01064/) (2022)
- [Student Success Prediction in MOOCs](https://mlsanity.com/paper/1711.06349/) (2017)
- [Analyzing Student Strategies In Blended Courses Using Clickstream Data](https://mlsanity.com/paper/2006.00421/) (2020)

## Built on by (most cited first, 7 of 7)

- [MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks](https://mlsanity.com/paper/2309.14118/) (2023)
- [The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations](https://mlsanity.com/paper/2307.00364/) (2023)
- [Intrinsic User-Centric Interpretability through Global Mixture of Experts](https://mlsanity.com/paper/2402.02933/) (2024)
- [Towards a Unified Framework for Evaluating Explanations](https://mlsanity.com/paper/2405.14016/) (2024)
- [ClickTree: A Tree-based Method for Predicting Math Students' Performance Based on Clickstream Data](https://mlsanity.com/paper/2403.14664/) (2024)
- [A constraints-based approach to fully interpretable neural networks for detecting learner behaviors](https://mlsanity.com/paper/2504.20055/) (2025)
- [Stable and Faithful Explanations for Knowledge Tracing](https://mlsanity.com/paper/2609.28502/) (2026)

## Abstract (arXiv)

Deep learning models for learning analytics have become increasingly popular over the last few years; however, these approaches are still not widely adopted in real-world settings, likely due to a lack of trust and transparency. In this paper, we tackle this issue by implementing explainable AI methods for black-box neural networks. This work focuses on the context of online and blended learning and the use case of student success prediction models. We use a pairwise study design, enabling us to investigate controlled differences between pairs of courses. Our analyses cover five course pairs that differ in one educationally relevant aspect and two popular instance-based explainable AI methods (LIME and SHAP). We quantitatively compare the distances between the explanations across courses and methods. We then validate the explanations of LIME and SHAP with 26 semi-structured interviews of university-level educators regarding which features they believe contribute most to student success, which explanations they trust most, and how they could transform these insights into actionable course design decisions. Our results show that quantitatively, explainers significantly disagree with each other about what is important, and qualitatively, experts themselves do not agree on which explanations are most trustworthy. All code, extended results, and the interview protocol are provided at https://github.com/epfl-ml4ed/trusting-explainers.