# The Grammar of Interactive Explanatory Model Analysis

arXiv: https://arxiv.org/abs/2005.00497 · HTML: https://mlsanity.com/paper/2005.00497/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2020.

## Built on by (most cited first, 24 of 24)

- [A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts](https://mlsanity.com/paper/2105.07190/) (2021)
- [Adversarial attacks and defenses in explainable artificial intelligence: A survey](https://mlsanity.com/paper/2306.06123/) (2023)
- [shapiq: Shapley Interactions for Machine Learning](https://mlsanity.com/paper/2410.01649/) (2024)
- [Position: Explain to Question not to Justify](https://mlsanity.com/paper/2402.13914/) (2024)
- [Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learning](https://mlsanity.com/paper/2501.14959/) (2025)
- [Exploration of the Rashomon Set Assists Trustworthy Explanations for Medical Data](https://mlsanity.com/paper/2308.11446/) (2023)
- [GraphXAIN: Narratives to Explain Graph Neural Networks](https://mlsanity.com/paper/2411.02540/) (2024)
- [Performance is not enough: the story told by a Rashomon quartet](https://mlsanity.com/paper/2302.13356/) (2023)
- [Interpretable machine learning for time-to-event prediction in medicine and healthcare](https://mlsanity.com/paper/2303.09817/) (2023)
- [Measuring User Understanding in Dialogue-based XAI Systems](https://mlsanity.com/paper/2408.06960/) (2024)
- [Aggregated Attributions for Explanatory Analysis of 3D Segmentation Models](https://mlsanity.com/paper/2407.16653/) (2024)
- [Explanation as a process: user-centric construction of multi-level and multi-modal explanations](https://mlsanity.com/paper/2110.03759/) (2021)
- [User-centric evaluation of explainability of AI with and for humans: a comprehensive empirical study](https://mlsanity.com/paper/2410.15952/) (2024)
- [Model Science: getting serious about verification, explanation and control of AI systems](https://mlsanity.com/paper/2508.20040/) (2025)
- [Explaining and visualizing black-box models through counterfactual paths](https://mlsanity.com/paper/2307.07764/) (2023)
- [On the Robustness of Global Feature Effect Explanations](https://mlsanity.com/paper/2406.09069/) (2024)
- [Interactive slice visualization for exploring machine learning models](https://mlsanity.com/paper/2101.06986/) (2021)
- [survex: an R package for explaining machine learning survival models](https://mlsanity.com/paper/2308.16113/) (2023)
- [Local Universal Explainer (LUX) -- a rule-based explainer with factual, counterfactual and visual explanations](https://mlsanity.com/paper/2310.14894/) (2023)
- [Data Science Principles for Interpretable and Explainable AI](https://mlsanity.com/paper/2405.10552/) (2024)
- [Do not explain without context: addressing the blind spot of model explanations](https://mlsanity.com/paper/2105.13787/) (2021)
- [Dataset resulting from the user study on comprehensibility of explainable AI algorithms](https://mlsanity.com/paper/2411.02419/) (2024)
- [Explanation User Interfaces: A Systematic Literature Review](https://mlsanity.com/paper/2505.20085/) (2025)
- [Beyond Additivity: Sparse Isotonic Shapley Regression toward Nonlinear Explainability](https://mlsanity.com/paper/2512.03112/) (2025)

## Abstract (arXiv)

The growing need for in-depth analysis of predictive models leads to a series of new methods for explaining their local and global properties. Which of these methods is the best? It turns out that this is an ill-posed question. One cannot sufficiently explain a black-box machine learning model using a single method that gives only one perspective. Isolated explanations are prone to misunderstanding, leading to wrong or simplistic reasoning. This problem is known as the Rashomon effect and refers to diverse, even contradictory, interpretations of the same phenomenon. Surprisingly, most methods developed for explainable and responsible machine learning focus on a single-aspect of the model behavior. In contrast, we showcase the problem of explainability as an interactive and sequential analysis of a model. This paper proposes how different Explanatory Model Analysis (EMA) methods complement each other and discusses why it is essential to juxtapose them. The introduced process of Interactive EMA (IEMA) derives from the algorithmic side of explainable machine learning and aims to embrace ideas developed in cognitive sciences. We formalize the grammar of IEMA to describe potential human-model dialogues. It is implemented in a widely used human-centered open-source software framework that adopts interactivity, customizability and automation as its main traits. We conduct a user study to evaluate the usefulness of IEMA, which indicates that an interactive sequential analysis of a model increases the performance and confidence of human decision making.