# A Unified Understanding of Deep NLP Models for Text Classification

arXiv: https://arxiv.org/abs/2206.09355 · HTML: https://mlsanity.com/paper/2206.09355/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 19 Jun 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters](https://mlsanity.com/paper/2002.01808/) (2020)
- [Visualizing and Understanding Neural Models in NLP](https://mlsanity.com/paper/1506.01066/) (2015)
- [Visualizing and Understanding Recurrent Networks](https://mlsanity.com/paper/1506.02078/) (2015)
- [Deep Learning Based Text Classification: A Comprehensive Review](https://mlsanity.com/paper/2004.03705/) (2020)
- [The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models](https://mlsanity.com/paper/2008.05122/) (2020)
- [Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers](https://mlsanity.com/paper/1801.06889/) (2018)
- [On Identifiability in Transformers](https://mlsanity.com/paper/1908.04211/) (2019)
- [Interpretable and Steerable Sequence Learning via Prototypes](https://mlsanity.com/paper/1907.09728/) (2019)
- [Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models](https://mlsanity.com/paper/1804.09299/) (2018)
- [RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records](https://mlsanity.com/paper/1805.10724/) (2018)
- [A Survey of Visual Analytics Techniques for Machine Learning](https://mlsanity.com/paper/2008.09632/) (2020)
- [Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Models](https://mlsanity.com/paper/1808.00196/) (2018)
- [Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models](https://mlsanity.com/paper/2009.07053/) (2020)
- [Understanding Hidden Memories of Recurrent Neural Networks](https://mlsanity.com/paper/1710.10777/) (2017)
- [Boundary thickness and robustness in learning models](https://mlsanity.com/paper/2007.05086/) (2020)
- [Visual Interaction with Deep Learning Models through Collaborative Semantic Inference](https://mlsanity.com/paper/1907.10739/) (2019)
- [RNNbow: Visualizing Learning via Backpropagation Gradients in Recurrent Neural Networks](https://mlsanity.com/paper/1907.12545/) (2019)

## Built on by (most cited first, 18 of 18)

- [Foundation Models Meet Visualizations: Challenges and Opportunities](https://mlsanity.com/paper/2310.05771/) (2023)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [Angler: Helping Machine Translation Practitioners Prioritize Model Improvements](https://mlsanity.com/paper/2304.05967/) (2023)
- [Evaluating Readability and Faithfulness of Concept-based Explanations](https://mlsanity.com/paper/2404.18533/) (2024)
- [A Unified Interactive Model Evaluation for Classification, Object Detection, and Instance Segmentation in Computer Vision](https://mlsanity.com/paper/2308.05168/) (2023)
- [JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models](https://mlsanity.com/paper/2404.08793/) (2024)
- [AttributionScanner: A Visual Analytics System for Model Validation with Metadata-Free Slice Finding](https://mlsanity.com/paper/2401.06462/) (2024)
- [Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles](https://mlsanity.com/paper/2208.09665/) (2022)
- [EL-VIT: Probing Vision Transformer with Interactive Visualization](https://mlsanity.com/paper/2401.12666/) (2024)
- [A Reliable Framework for Human-in-the-Loop Anomaly Detection in Time Series](https://mlsanity.com/paper/2405.03234/) (2024)
- [CommonsenseVIS: Visualizing and Understanding Commonsense Reasoning Capabilities of Natural Language Models](https://mlsanity.com/paper/2307.12382/) (2023)
- [JailbreakHunter: A Visual Analytics Approach for Jailbreak Prompts Discovery from Large-Scale Human-LLM Conversational Datasets](https://mlsanity.com/paper/2407.03045/) (2024)
- [Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models](https://mlsanity.com/paper/2408.11827/) (2024)
- [iGAiVA: Integrated Generative AI and Visual Analytics in a Machine Learning Workflow for Text Classification](https://mlsanity.com/paper/2409.15848/) (2024)
- [POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models](https://mlsanity.com/paper/2406.03843/) (2024)
- [NeuroBreak: Unveil Internal Jailbreak Mechanisms in Large Language Models](https://mlsanity.com/paper/2509.03985/) (2025)
- [Mixed-Type Wafer Classification For Low Memory Devices Using Knowledge Distillation](https://mlsanity.com/paper/2303.13974/) (2023)
- [Visual Analytics for Fine-grained Text Classification Models and Datasets](https://mlsanity.com/paper/2403.15492/) (2024)

## Abstract (arXiv)

The rapid development of deep natural language processing (NLP) models for text classification has led to an urgent need for a unified understanding of these models proposed individually. Existing methods cannot meet the need for understanding different models in one framework due to the lack of a unified measure for explaining both low-level (e.g., words) and high-level (e.g., phrases) features. We have developed a visual analysis tool, DeepNLPVis, to enable a unified understanding of NLP models for text classification. The key idea is a mutual information-based measure, which provides quantitative explanations on how each layer of a model maintains the information of input words in a sample. We model the intra- and inter-word information at each layer measuring the importance of a word to the final prediction as well as the relationships between words, such as the formation of phrases. A multi-level visualization, which consists of a corpus-level, a sample-level, and a word-level visualization, supports the analysis from the overall training set to individual samples. Two case studies on classification tasks and comparison between models demonstrate that DeepNLPVis can help users effectively identify potential problems caused by samples and model architectures and then make informed improvements.