mlsanityData to 7 Oct 2026

exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models

arXiv 1910.05276 ↗First posted 11 Oct 2019cs.CL cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

Large language models can produce powerful contextual representations that lead to improvements across many NLP tasks. Since these models are typically guided by a sequence of learned self attention mechanisms and may comprise undesired inductive biases, it is paramount to be able to explore what the attention has learned. While static analyses of these models lead to targeted insights, interactive tools are more dynamic and can help humans better gain an intuition for the model-internal reasoning process. We present exBERT, an interactive tool named after the popular BERT language model, that provides insights into the meaning of the contextual representations by matching a human-specified input to similar contexts in a large annotated dataset. By aggregating the annotations of the matching similar contexts, exBERT helps intuitively explain what each attention-head has learned.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 9; 2023: 6; 2024: 7; 2025: 3; 2026: 1.

33 papers.

Sort:

By area:

  1. 2021Benchmarking and Survey of Explanation Methods for Black Box Modelscited by 136
  2. 2023Eliciting Latent Knowledge from Quirky Language Modelscited by 56
  3. 2021Explanation-Based Human Debugging of NLP Models: A Surveycited by 53
  4. 2022VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformerscited by 53
  5. 2022What Do They Capture? -- A Structural Analysis of Pre-Trained Language Models for Source Codecited by 50
  6. 2023AttentionViz: A Global View of Transformer Attentioncited by 50
  7. 2022On the Explainability of Natural Language Processing Deep Modelscited by 37
  8. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  9. 2021Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behaviorcited by 29
  10. 2022LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Modelscited by 28
  11. 2022Visual Comparison of Language Model Adaptationcited by 14
  12. 2021GenNI: Human-AI Collaboration for Data-Backed Text Generationcited by 11
21 more
  1. 2024Neuron-based Personality Trait Induction in Large Language Modelscited by 9
  2. 2022Interpreting BERT-based Text Similarity via Activation and Saliency Mapscited by 8
  3. 2021RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualizationcited by 6
  4. 2024Talaria: Interactively Optimizing Machine Learning Models for Efficient Inferencecited by 6
  5. 2024Explainable AI for Mental Disorder Detection via Social Media: A survey and outlookcited by 6
  6. 2022It Takes Two Flints to Make a Fire: Multitask Learning of Neural Relation and Explanation Classifierscited by 5
  7. 2023DeepLens: Interactive Out-of-distribution Data Detection in NLP Modelscited by 5
  8. 2024Procedural Adherence and Interpretability Through Neuro-Symbolic Generative Agentscited by 5
  9. 2024Lightweight Safety Classification Using Pruned Language Modelscited by 5
  10. 2025Reasoning-Grounded Natural Language Explanations for Language Modelscited by 4
  11. 2025Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformerscited by 4
  12. 2022Astroconformer: Inferring Surface Gravity of Stars from Stellar Light Curves with Transformercited by 3
  13. 2023Visual Analytics for Generative Transformer Modelscited by 2
  14. 2022Does Attention Mechanism Possess the Feature of Human Reading? A Perspective of Sentiment Classification Taskcited by 1
  15. 2023Opening the Black Box: Analyzing Attention Weights and Hidden States in Pre-trained Language Models for Non-language Taskscited by 1
  16. 2024Consistency Matters: Explore LLMs Consistency From a Black-Box Perspectivecited by 1
  17. 2024A Theoretical Survey on Foundation Modelscited by 1
  18. 2021Representing ELMo embeddings as two-dimensional text onlinecited by 0
  19. 2021Context-Sensitive Visualization of Deep Learning Natural Language Processing Modelscited by 0
  20. 2023Towards Explainable AI Writing Assistants for Non-native English Speakerscited by 0
  21. 2026string2string Studio: An Interactive, In-Browser Platform for String-to-String Algorithmscited by 0