# AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling

arXiv: https://arxiv.org/abs/1807.00154 · HTML: https://mlsanity.com/paper/1807.00154/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2018.

## Built on by (most cited first, 18 of 18)

- [Evaluating the Explainers: Black-Box Explainable Machine Learning for Student Success Prediction in MOOCs](https://mlsanity.com/paper/2207.00551/) (2022)
- [The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations](https://mlsanity.com/paper/2307.00364/) (2023)
- [Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education?](https://mlsanity.com/paper/2104.12920/) (2021)
- [Intrinsic User-Centric Interpretability through Global Mixture of Experts](https://mlsanity.com/paper/2402.02933/) (2024)
- [RIPPLE: Concept-Based Interpretation for Raw Time Series Models in Education](https://mlsanity.com/paper/2212.01133/) (2022)
- [Trusting the Explainers: Teacher Validation of Explainable Artificial Intelligence for Course Design](https://mlsanity.com/paper/2212.08955/) (2022)
- [AGI: Artificial General Intelligence for Education](https://mlsanity.com/paper/2304.12479/) (2023)
- [The Ethics of AI in Education](https://mlsanity.com/paper/2406.11842/) (2024)
- [Augmenting deep neural networks with symbolic knowledge: Towards trustworthy and interpretable AI for education](https://mlsanity.com/paper/2311.00393/) (2023)
- [Towards Responsible and Trustworthy Educational Data Mining: Comparing Symbolic, Sub-Symbolic, and Neural-Symbolic AI Methods](https://mlsanity.com/paper/2504.00615/) (2025)
- [Towards responsible AI for education: Hybrid human-AI to confront the Elephant in the room](https://mlsanity.com/paper/2504.16148/) (2025)
- [Absolute Value Constraint: The Reason for Invalid Performance Evaluation Results of Neural Network Models for Stock Price Prediction](https://mlsanity.com/paper/2101.10942/) (2021)
- [Towards a Unified Framework for Evaluating Explanations](https://mlsanity.com/paper/2405.14016/) (2024)
- [Problems With Large Language Models for Learner Modelling: Why LLMs Alone Fall Short for Responsible Tutoring in K--12 Education](https://mlsanity.com/paper/2512.23036/) (2025)
- [Generalisable Methods for Early Prediction in Interactive Simulations for Education](https://mlsanity.com/paper/2207.01457/) (2022)
- [Lost in Translation: Reimagining the Machine Learning Life Cycle in Education](https://mlsanity.com/paper/2209.03929/) (2022)
- [From Algorithm Worship to the Art of Human Learning: Insights from 50-year journey of AI in Education](https://mlsanity.com/paper/2403.05544/) (2024)
- [A Computational Model of Inclusive Pedagogy: From Understanding to Application](https://mlsanity.com/paper/2505.02853/) (2025)

## Abstract (arXiv)

Interpretability of the underlying AI representations is a key raison d'etre for Open Learner Modelling (OLM) -- a branch of Intelligent Tutoring Systems (ITS) research. OLMs provide tools for 'opening' up the AI models of learners' cognition and emotions for the purpose of supporting human learning and teaching. Over thirty years of research in ITS (also known as AI in Education) produced important work, which informs about how AI can be used in Education to best effects and, through the OLM research, what are the necessary considerations to make it interpretable and explainable for the benefit of learning. We argue that this work can provide a valuable starting point for a framework of interpretable AI, and as such is of relevance to the application of both knowledge-based and machine learning systems in other high-stakes contexts, beyond education.