# Visualizing Attention in Transformer-Based Language Representation Models

arXiv: https://arxiv.org/abs/1904.02679 · HTML: https://mlsanity.com/paper/1904.02679/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2019.

## Built on by (most cited first, 8 of 8)

- [A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models](https://mlsanity.com/paper/2502.17516/) (2025)
- [InternalInspector I2: Robust Confidence Estimation in LLMs through Internal States](https://mlsanity.com/paper/2406.12053/) (2024)
- [Interpretability Illusions in the Generalization of Simplified Models](https://mlsanity.com/paper/2312.03656/) (2023)
- [ReasonGraph: Visualisation of Reasoning Paths](https://mlsanity.com/paper/2503.03979/) (2025)
- [Lines of Thought in Large Language Models](https://mlsanity.com/paper/2410.01545/) (2024)
- [ReTrace: Interactive Visualizations for Reasoning Traces of Large Reasoning Models](https://mlsanity.com/paper/2511.11187/) (2025)
- [Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning](https://mlsanity.com/paper/2607.07316/) (2026)
- [Extracting Interpretable Local and Global Representations from Attention on Time Series](https://mlsanity.com/paper/2312.11466/) (2023)

## Abstract (arXiv)

We present an open-source tool for visualizing multi-head self-attention in Transformer-based language representation models. The tool extends earlier work by visualizing attention at three levels of granularity: the attention-head level, the model level, and the neuron level. We describe how each of these views can help to interpret the model, and we demonstrate the tool on the BERT model and the OpenAI GPT-2 model. We also present three use cases for analyzing GPT-2: detecting model bias, identifying recurring patterns, and linking neurons to model behavior.