# Towards a Unified Framework for Evaluating Explanations

arXiv: https://arxiv.org/abs/2405.14016 · HTML: https://mlsanity.com/paper/2405.14016/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 13 of 13)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Questioning the AI: Informing Design Practices for Explainable AI User Experiences](https://mlsanity.com/paper/2001.02478/) (2020)
- [Human-Centered Explainable AI (XAI): From Algorithms to User Experiences](https://mlsanity.com/paper/2110.10790/) (2021)
- [On quantitative aspects of model interpretability](https://mlsanity.com/paper/2007.07584/) (2020)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [The Challenge of Crafting Intelligible Intelligence](https://mlsanity.com/paper/1803.04263/) (2018)
- [AI in Education needs interpretable machine learning: Lessons from Open Learner Modelling](https://mlsanity.com/paper/1807.00154/) (2018)
- [The future of human-centric eXplainable Artificial Intelligence (XAI) is not post-hoc explanations](https://mlsanity.com/paper/2307.00364/) (2023)
- [Trusting the Explainers: Teacher Validation of Explainable Artificial Intelligence for Course Design](https://mlsanity.com/paper/2212.08955/) (2022)

## Built on by (most cited first, 2 of 2)

- [A constraints-based approach to fully interpretable neural networks for detecting learner behaviors](https://mlsanity.com/paper/2504.20055/) (2025)
- [RegKT: Interpretable and Robust Deep Knowledge Tracing With IRT-Regularizer](https://mlsanity.com/paper/2609.21791/) (2026)

## Abstract (arXiv)

The challenge of creating interpretable models has been taken up by two main research communities: ML researchers primarily focused on lower-level explainability methods that suit the needs of engineers, and HCI researchers who have more heavily emphasized user-centered approaches often based on participatory design methods. This paper reviews how these communities have evaluated interpretability, identifying overlaps and semantic misalignments. We propose moving towards a unified framework of evaluation criteria and lay the groundwork for such a framework by articulating the relationships between existing criteria. We argue that explanations serve as mediators between models and stakeholders, whether for intrinsically interpretable models or opaque black-box models analyzed via post-hoc techniques. We further argue that useful explanations require both faithfulness and intelligibility. Explanation plausibility is a prerequisite for intelligibility, while stability is a prerequisite for explanation faithfulness. We illustrate these criteria, as well as specific evaluation methods, using examples from an ongoing study of an interpretable neural network for predicting a particular learner behavior.