# Investigating the Duality of Interpretability and Explainability in Machine Learning

arXiv: https://arxiv.org/abs/2503.21356 · HTML: https://mlsanity.com/paper/2503.21356/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 10 of 10)

- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [The many Shapley values for model explanation](https://mlsanity.com/paper/1908.08474/) (2019)
- [Metrics for Explainable AI: Challenges and Prospects](https://mlsanity.com/paper/1812.04608/) (2018)
- [On the Tractability of SHAP Explanations](https://mlsanity.com/paper/2009.08634/) (2020)
- [Counterfactuals and Causability in Explainable Artificial Intelligence: Theory, Algorithms, and Applications](https://mlsanity.com/paper/2103.04244/) (2021)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?](https://mlsanity.com/paper/2304.04780/) (2023)
- [Pitfalls of Explainable ML: An Industry Perspective](https://mlsanity.com/paper/2106.07758/) (2021)

## Built on by (most cited first, 8 of 8)

- [Surrogate Modeling and Explainable Artificial Intelligence for Complex Systems: A Workflow for Automated Simulation Exploration](https://mlsanity.com/paper/2510.16742/) (2025)
- [On the definition and importance of interpretability in scientific machine learning](https://mlsanity.com/paper/2505.13510/) (2025)
- [Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations](https://mlsanity.com/paper/2510.17256/) (2025)
- [An experimental survey and Perspective View on Meta-Learning for Automated Algorithms Selection and Parametrization](https://mlsanity.com/paper/2504.06207/) (2025)
- [From Black-Box Tuning to Guided Optimization via Hyperparameters Interaction Analysis](https://mlsanity.com/paper/2512.19246/) (2025)
- [Fully Geometric Multi-Hop Reasoning on Knowledge Graphs with Transitive Relations](https://mlsanity.com/paper/2505.12369/) (2025)
- [Exploring Content and Social Connections of Fake News with Explainable Text and Graph Learning](https://mlsanity.com/paper/2508.10040/) (2025)
- [Improving Neural Retrieval with Attribution-Guided Query Rewriting](https://mlsanity.com/paper/2602.11841/) (2026)

## Abstract (arXiv)

The rapid evolution of machine learning (ML) has led to the widespread adoption of complex "black box" models, such as deep neural networks and ensemble methods. These models exhibit exceptional predictive performance, making them invaluable for critical decision-making across diverse domains within society. However, their inherently opaque nature raises concerns about transparency and interpretability, making them untrustworthy decision support systems. To alleviate such a barrier to high-stakes adoption, research community focus has been on developing methods to explain black box models as a means to address the challenges they pose. Efforts are focused on explaining these models instead of developing ones that are inherently interpretable. Designing inherently interpretable models from the outset, however, can pave the path towards responsible and beneficial applications in the field of ML. In this position paper, we clarify the chasm between explaining black boxes and adopting inherently interpretable models. We emphasize the imperative need for model interpretability and, following the purpose of attaining better (i.e., more effective or efficient w.r.t. predictive performance) and trustworthy predictors, provide an experimental evaluation of latest hybrid learning methods that integrates symbolic knowledge into neural network predictors. We demonstrate how interpretable hybrid models could potentially supplant black box ones in different domains.