# Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models

arXiv: https://arxiv.org/abs/1808.00196 · HTML: https://mlsanity.com/paper/1808.00196/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2018.

## Built on by (most cited first, 30 of 50)

- [Salient ImageNet: How to discover spurious features in Deep Learning?](https://mlsanity.com/paper/2110.04301/) (2021)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [A Multimodal Automated Interpretability Agent](https://mlsanity.com/paper/2404.14394/) (2024)
- [MM-SpuBench: Towards Better Understanding of Spurious Biases in Multimodal LLMs](https://mlsanity.com/paper/2406.17126/) (2024)
- [M2Lens: Visualizing and Explaining Multimodal Models for Sentiment Analysis](https://mlsanity.com/paper/2107.08264/) (2021)
- [The State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations](https://mlsanity.com/paper/2212.11737/) (2022)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [Sibyl: Understanding and Addressing the Usability Challenges of Machine Learning In High-Stakes Decision Making](https://mlsanity.com/paper/2103.02071/) (2021)
- [A Unified Understanding of Deep NLP Models for Text Classification](https://mlsanity.com/paper/2206.09355/) (2022)
- [Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels](https://mlsanity.com/paper/2110.12536/) (2021)
- [Core Risk Minimization using Salient ImageNet](https://mlsanity.com/paper/2203.15566/) (2022)
- [Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models](https://mlsanity.com/paper/2102.00036/) (2021)
- [Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML](https://mlsanity.com/paper/2104.04375/) (2021)
- [VAC-CNN: A Visual Analytics System for Comparative Studies of Deep Convolutional Neural Networks](https://mlsanity.com/paper/2110.13252/) (2021)
- [QualEval: Qualitative Evaluation for Model Improvement](https://mlsanity.com/paper/2311.02807/) (2023)
- [Learning-From-Disagreement: A Model Comparison and Visual Analytics Framework](https://mlsanity.com/paper/2201.07849/) (2022)
- [Learning Robust Classifiers with Self-Guided Spurious Correlation Mitigation](https://mlsanity.com/paper/2405.03649/) (2024)
- [VisRuler: Visual Analytics for Extracting Decision Rules from Bagged and Boosted Decision Trees](https://mlsanity.com/paper/2112.00334/) (2021)
- [Class-constrained t-SNE: Combining Data Features and Class Probabilities](https://mlsanity.com/paper/2308.13837/) (2023)
- [Spuriousness-Aware Meta-Learning for Learning Robust Classifiers](https://mlsanity.com/paper/2406.10742/) (2024)
- [Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles](https://mlsanity.com/paper/2103.10229/) (2021)
- [Model Doctor: A Simple Gradient Aggregation Strategy for Diagnosing and Treating CNN Classifiers](https://mlsanity.com/paper/2112.04934/) (2021)
- [Visual Auditor: Interactive Visualization for Detection and Summarization of Model Biases](https://mlsanity.com/paper/2206.12540/) (2022)
- [Benchmarking Spurious Bias in Few-Shot Image Classifiers](https://mlsanity.com/paper/2409.02882/) (2024)
- [Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks](https://mlsanity.com/paper/2502.10526/) (2025)
- [FedNLP: An interpretable NLP System to Decode Federal Reserve Communications](https://mlsanity.com/paper/2106.06247/) (2021)
- [Unlearning Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methods](https://mlsanity.com/paper/2508.12730/) (2025)
- [iSEA: An Interactive Pipeline for Semantic Error Analysis of NLP Models](https://mlsanity.com/paper/2203.04408/) (2022)
- [Data-Centric Debugging: mitigating model failures via targeted data collection](https://mlsanity.com/paper/2211.09859/) (2022)
- [Identifying and Mitigating Model Failures through Few-shot CLIP-aided Diffusion Generation](https://mlsanity.com/paper/2312.05464/) (2023)

## Abstract (arXiv)

Interpretation and diagnosis of machine learning models have gained renewed interest in recent years with breakthroughs in new approaches. We present Manifold, a framework that utilizes visual analysis techniques to support interpretation, debugging, and comparison of machine learning models in a more transparent and interactive manner. Conventional techniques usually focus on visualizing the internal logic of a specific model type (i.e., deep neural networks), lacking the ability to extend to a more complex scenario where different model types are integrated. To this end, Manifold is designed as a generic framework that does not rely on or access the internal logic of the model and solely observes the input (i.e., instances or features) and the output (i.e., the predicted result and probability distribution). We describe the workflow of Manifold as an iterative process consisting of three major phases that are commonly involved in the model development and diagnosis process: inspection (hypothesis), explanation (reasoning), and refinement (verification). The visual components supporting these tasks include a scatterplot-based visual summary that overviews the models' outcome and a customizable tabular view that reveals feature discrimination. We demonstrate current applications of the framework on the classification and regression tasks and discuss other potential machine learning use scenarios where Manifold can be applied.