# Towards responsible AI for education: Hybrid human-AI to confront the Elephant in the room arXiv: https://arxiv.org/abs/2504.16148 ยท HTML: https://mlsanity.com/paper/2504.16148/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Builds on 41 of its 206 references that are in the map. - First posted on arXiv 22 Apr 2025. ## Builds on (most cited first, 30 of 41) - [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016) - [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017) - [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022) - [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019) - [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017) - [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019) - [Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods](https://mlsanity.com/paper/1911.02508/) (2019) - [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017) - [The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision](https://mlsanity.com/paper/1904.12584/) (2019) - [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016) - [Deep Knowledge Tracing](https://mlsanity.com/paper/1506.05908/) (2015) - [Interpretable machine learning: definitions, methods, and applications](https://mlsanity.com/paper/1901.04592/) (2019) - [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019) - [Model-Agnostic Interpretability of Machine Learning](https://mlsanity.com/paper/1606.05386/) (2016) - [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020) - [Neural-Symbolic Learning and Reasoning: A Survey and Interpretation](https://mlsanity.com/paper/1711.03902/) (2017) - [Teaching Large Language Models to Reason with Reinforcement Learning](https://mlsanity.com/paper/2403.04642/) (2024) - [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022) - [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024) - [On the Tractability of SHAP Explanations](https://mlsanity.com/paper/2009.08634/) (2020) - [Explaining the Explainer: A First Theoretical Analysis of LIME](https://mlsanity.com/paper/2001.03447/) (2020) - [Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review](https://mlsanity.com/paper/2303.13379/) (2023) - [Knowledge Tracing: A Survey](https://mlsanity.com/paper/2201.06953/) (2022) - [Incorporating Domain Knowledge into Deep Neural Networks](https://mlsanity.com/paper/2103.00180/) (2021) - [Addressing Two Problems in Deep Knowledge Tracing via Prediction-Consistent Regularization](https://mlsanity.com/paper/1806.02180/) (2018) - [Fine-tuning ChatGPT for Automatic Scoring](https://mlsanity.com/paper/2310.10072/) (2023) - [Measurable Counterfactual Local Explanations for Any Classifier](https://mlsanity.com/paper/1908.03020/) (2019) - [Preference Learning Algorithms Do Not Learn Preference Rankings](https://mlsanity.com/paper/2405.19534/) (2024) - [Defining Locality for Surrogates in Post-hoc Interpretablity](https://mlsanity.com/paper/1806.07498/) (2018) - [Evaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester-Long Field Study](https://mlsanity.com/paper/2404.13414/) (2024) ## Built on by (most cited first, 3 of 3) - [Problems With Large Language Models for Learner Modelling: Why LLMs Alone Fall Short for Responsible Tutoring in K--12 Education](https://mlsanity.com/paper/2512.23036/) (2025) - [Neural-Symbolic Knowledge Tracing: Injecting Educational Knowledge into Deep Learning for Responsible Learner Modelling](https://mlsanity.com/paper/2604.08263/) (2026) - [Aurora: Neuro-Symbolic AI Driven Advising Agent](https://mlsanity.com/paper/2602.17999/) (2026) ## Abstract (arXiv) Despite significant advancements in AI-driven educational systems and ongoing calls for responsible AI for education, several critical issues remain unresolved -- acting as the elephant in the room within AI in education, learning analytics, educational data mining, learning sciences, and educational psychology communities. This critical analysis identifies and examines nine persistent challenges that continue to undermine the fairness, transparency, and effectiveness of current AI methods and applications in education. These include: (1) the lack of clarity around what AI for education truly means -- often ignoring the distinct purposes, strengths, and limitations of different AI families -- and the trend of equating it with domain-agnostic, company-driven large language models; (2) the widespread neglect of essential learning processes such as motivation, emotion, and (meta)cognition in AI-driven learner modelling and their contextual nature; (3) limited integration of domain knowledge and lack of stakeholder involvement in AI design and development; (4) continued use of non-sequential machine learning models on temporal educational data; (5) misuse of non-sequential metrics to evaluate sequential models; (6) use of unreliable explainable AI methods to provide explanations for black-box models; (7) ignoring ethical guidelines in addressing data inconsistencies during model training; (8) use of mainstream AI methods for pattern discovery and learning analytics without systematic benchmarking; and (9) overemphasis on global prescriptions while overlooking localised, student-specific recommendations. Supported by theoretical and empirical research, we demonstrate how hybrid AI methods -- specifically neural-symbolic AI -- can address the elephant in the room and serve as the foundation for responsible, trustworthy AI systems in education.