# explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

arXiv: https://arxiv.org/abs/1908.00087 · HTML: https://mlsanity.com/paper/1908.00087/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2019.

## Built on by (most cited first, 30 of 70)

- [Expanding Explainability: Towards Social Transparency in AI systems](https://mlsanity.com/paper/2101.04719/) (2021)
- [A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts](https://mlsanity.com/paper/2105.07190/) (2021)
- [Beyond Expertise and Roles: A Framework to Characterize the Stakeholders of Interpretable Machine Learning and their Needs](https://mlsanity.com/paper/2101.09824/) (2021)
- [The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations](https://mlsanity.com/paper/2212.11737/) (2022)
- [TS-MULE: Local Interpretable Model-Agnostic Explanations for Time Series Forecast Models](https://mlsanity.com/paper/2109.08438/) (2021)
- [Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant](https://mlsanity.com/paper/2501.17546/) (2025)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [RELIC: Investigating Large Language Model Responses using Self-Consistency](https://mlsanity.com/paper/2311.16842/) (2023)
- [Position: Explain to Question not to Justify](https://mlsanity.com/paper/2402.13914/) (2024)
- [Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making](https://mlsanity.com/paper/2103.02071/) (2021)
- [Directive Explanations for Monitoring the Risk of Diabetes Onset: Introducing Directive Data-Centric Explanations and Combinations to Support What-If Explorations](https://mlsanity.com/paper/2302.10671/) (2023)
- [A Survey on AI Assurance](https://mlsanity.com/paper/2111.07505/) (2021)
- [EXMOS: Explanatory Model Steering Through Multifaceted Explanations and Data Configurations](https://mlsanity.com/paper/2402.00491/) (2024)
- [Angler: Helping Machine Translation Practitioners Prioritize Model Improvements](https://mlsanity.com/paper/2304.05967/) (2023)
- [Explainability in reinforcement learning: perspective and position](https://mlsanity.com/paper/2203.11547/) (2022)
- [Beware the Rationalization Trap! When Language Model Explainability Diverges from our Mental Models of Language](https://mlsanity.com/paper/2207.06897/) (2022)
- [A Survey on Explainable Artificial Intelligence for Cybersecurity](https://mlsanity.com/paper/2303.12942/) (2023)
- [Time Series Model Attribution Visualizations as Explanations](https://mlsanity.com/paper/2109.12935/) (2021)
- [XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning](https://mlsanity.com/paper/2202.11954/) (2022)
- [Which LIME should I trust? Concepts, Challenges, and Solutions](https://mlsanity.com/paper/2503.24365/) (2025)
- [JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models](https://mlsanity.com/paper/2404.08793/) (2024)
- [Preventing Machine Learning Poisoning Attacks Using Authentication and Provenance](https://mlsanity.com/paper/2105.10051/) (2021)
- [Interactive Explanation with Varying Level of Details in an Explainable Scientific Literature Recommender System](https://mlsanity.com/paper/2306.05809/) (2023)
- [Visual Explanations with Attributions and Counterfactuals on Time Series Classification](https://mlsanity.com/paper/2307.08494/) (2023)
- [ProtoExplorer: Interpretable Forensic Analysis of Deepfake Videos using Prototype Exploration and Refinement](https://mlsanity.com/paper/2309.11155/) (2023)
- [generAItor: Tree-in-the-Loop Text Generation for Language Model Explainability and Adaptation](https://mlsanity.com/paper/2403.07627/) (2024)
- [The FIX Benchmark: Extracting Features Interpretable to eXperts](https://mlsanity.com/paper/2409.13684/) (2024)
- [Contrastive Identification of Covariate Shift in Image Data](https://mlsanity.com/paper/2108.08000/) (2021)
- [Interactive Analysis of CNN Robustness](https://mlsanity.com/paper/2110.07667/) (2021)
- [Justification vs. Transparency: Why and How Visual Explanations in a Scientific Literature Recommender System](https://mlsanity.com/paper/2305.17034/) (2023)

## Abstract (arXiv)

We propose a framework for interactive and explainable machine learning that enables users to (1) understand machine learning models; (2) diagnose model limitations using different explainable AI methods; as well as (3) refine and optimize the models. Our framework combines an iterative XAI pipeline with eight global monitoring and steering mechanisms, including quality monitoring, provenance tracking, model comparison, and trust building. To operationalize the framework, we present explAIner, a visual analytics system for interactive and explainable machine learning that instantiates all phases of the suggested pipeline within the commonly used TensorBoard environment. We performed a user-study with nine participants across different expertise levels to examine their perception of our workflow and to collect suggestions to fill the gap between our system and framework. The evaluation confirms that our tightly integrated system leads to an informed machine learning process while disclosing opportunities for further extensions.