# A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations

arXiv: https://arxiv.org/abs/1705.01968 · HTML: https://mlsanity.com/paper/1705.01968/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2017.

## Built on by (most cited first, 20 of 20)

- [The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations](https://mlsanity.com/paper/2212.11737/) (2022)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [AdViCE: Aggregated Visual Counterfactual Explanations for Machine Learning Model Validation](https://mlsanity.com/paper/2109.05629/) (2021)
- [Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles](https://mlsanity.com/paper/2103.10229/) (2021)
- [Model Doctor: A Simple Gradient Aggregation Strategy for Diagnosing and Treating CNN Classifiers](https://mlsanity.com/paper/2112.04934/) (2021)
- [Interactive Machine Learning: A State of the Art Review](https://mlsanity.com/paper/2207.06196/) (2022)
- [Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts](https://mlsanity.com/paper/2205.05749/) (2022)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [Designing a Direct Feedback Loop between Humans and Convolutional Neural Networks through Local Explanations](https://mlsanity.com/paper/2307.04036/) (2023)
- [Logic-based Explanations for Linear Support Vector Classifiers with Reject Option](https://mlsanity.com/paper/2403.16190/) (2024)
- [ESCAPE: Countering Systematic Errors from Machine's Blind Spots via Interactive Visual Analysis](https://mlsanity.com/paper/2303.09657/) (2023)
- [ATWL: A Formal Language for Representing, Comparing, and Reusing Visual Analytics Workflows](https://mlsanity.com/paper/2605.25489/) (2026)
- [Black-box Error Diagnosis in Deep Neural Networks for Computer Vision: a Survey of Tools](https://mlsanity.com/paper/2201.06444/) (2022)
- [GeoExplainer: A Visual Analytics Framework for Spatial Modeling Contextualization and Report Generation](https://mlsanity.com/paper/2308.13588/) (2023)
- [TRIVEA: Transparent Ranking Interpretation using Visual Explanation of Black-Box Algorithmic Rankers](https://mlsanity.com/paper/2308.14622/) (2023)
- [ViT-Calibrator: Decision Stream Calibration for Vision Transformer](https://mlsanity.com/paper/2304.04354/) (2023)
- [TA3: Testing Against Adversarial Attacks on Machine Learning Models](https://mlsanity.com/paper/2410.05334/) (2024)
- [MLMC: Interactive multi-label multi-classifier evaluation without confusion matrices](https://mlsanity.com/paper/2501.14460/) (2025)
- [Explanation User Interfaces: A Systematic Literature Review](https://mlsanity.com/paper/2505.20085/) (2025)
- [SketchXplain: Intuitive Visual Explanations of Image Classifiers with Sketches](https://mlsanity.com/paper/2606.17646/) (2026)

## Abstract (arXiv)

Human-in-the-loop data analysis applications necessitate greater transparency in machine learning models for experts to understand and trust their decisions. To this end, we propose a visual analytics workflow to help data scientists and domain experts explore, diagnose, and understand the decisions made by a binary classifier. The approach leverages "instance-level explanations", measures of local feature relevance that explain single instances, and uses them to build a set of visual representations that guide the users in their investigation. The workflow is based on three main visual representations and steps: one based on aggregate statistics to see how data distributes across correct / incorrect decisions; one based on explanations to understand which features are used to make these decisions; and one based on raw data, to derive insights on potential root causes for the observed patterns. The workflow is derived from a long-term collaboration with a group of machine learning and healthcare professionals who used our method to make sense of machine learning models they developed. The case study from this collaboration demonstrates that the proposed workflow helps experts derive useful knowledge about the model and the phenomena it describes, thus experts can generate useful hypotheses on how a model can be improved.