# Challenges in Evaluating Explanation Methods for Static and Evolving Data

arXiv: https://arxiv.org/abs/2608.06351 · HTML: https://mlsanity.com/paper/2608.06351/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Learning under Concept Drift: A Review](https://mlsanity.com/paper/2004.05785/) (2020)
- [From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation](https://mlsanity.com/paper/2206.03208/) (2022)
- [GANterfactual - Counterfactual Explanations for Medical Non-Experts using Generative Adversarial Learning](https://mlsanity.com/paper/2012.11905/) (2020)
- [Model Based Explanations of Concept Drift](https://mlsanity.com/paper/2303.09331/) (2023)
- [Probabilistically Plausible Counterfactual Explanations with Normalizing Flows](https://mlsanity.com/paper/2405.17640/) (2024)
- [A multi-criteria approach for selecting an explanation from the set of counterfactuals produced by an ensemble of explainers](https://mlsanity.com/paper/2403.13940/) (2024)
- [GLANCE: Global Actions in a Nutshell for Counterfactual Explainability](https://mlsanity.com/paper/2405.18921/) (2024)
- [Learning from Data Streams: An Overview and Update](https://mlsanity.com/paper/2212.14720/) (2022)
- [Explaining Concept Drift through the Evolution of Group Counterfactuals](https://mlsanity.com/paper/2509.09616/) (2025)
- [Position: Explainability Research Must Prioritize Foundations over Ad-hoc Methods](https://mlsanity.com/paper/2607.14123/) (2026)
- [DetoxAI: a Python Toolkit for Debiasing Deep Learning Models in Computer Vision](https://mlsanity.com/paper/2505.05492/) (2025)

## Abstract (arXiv)

This paper addresses the limitations of Explainable Artificial Intelligence (XAI) with respect to insufficient evaluation. They are illustrated through the DetoxAI image recognition system for bias detection and concept unlearning. Then, an example of a human-grounded evaluation of methods for explaining image classification is presented. The paper further explores methods for adapting explanations to evolving data streams with concept drift. Experiences with adapting counterfactuals for this problem are discussed. Finally it is related to the challenges of tracking the co-evolution of data, models, and explanations.footnoteThis paper has been accepted for a publication in J.Nalepa (ed) Explainable AI in Space. Proceedings of EASi 2026 Workshop at IJCAI-ECAI 2026 Bremen, Springer CCIS vol 3107 (2016).