mlsanityData to 14 Oct 2026

Visualizing and Explaining Language Models

arXiv 2205.10238 ↗First posted 30 Apr 2022cs.CL cs.HC cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

During the last decade, Natural Language Processing has become, after Computer Vision, the second field of Artificial Intelligence that was massively changed by the advent of Deep Learning. Regardless of the architecture, the language models of the day need to be able to process or generate text, as well as predict missing words, sentences or relations depending on the task. Due to their black-box nature, such models are difficult to interpret and explain to third parties. Visualization is often the bridge that language model designers use to explain their work, as the coloring of the salient words and phrases, clustering or neuron activations can be used to quickly understand the underlying models. This paper showcases the techniques used in some of the most popular Deep Learning for NLP visualizations, with a special focus on interpretability and explainability.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 77 of its 111 references that are in the map. First posted on arXiv 30 Apr 2022.

Builds on

77 of 111 references.

References · 77

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  6. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  9. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  10. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  11. 2020Extracting Training Data from Large Language Modelscited by 2,189
  12. 2020Reformer: The Efficient Transformercited by 1,668
38 more
  1. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  2. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,230
  3. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  4. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  5. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  6. 2019Attention is not Explanationcited by 929
  7. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 856
  8. 2020Quantifying Attention Flow in Transformerscited by 775
  9. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  10. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  11. 2019Attention is not not Explanationcited by 636
  12. 2021Transformer in Transformercited by 633
  13. 2019ELI5: Long Form Question Answeringcited by 575
  14. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  15. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 457
  16. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 443
  17. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  18. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  19. 2020Neural Additive Models: Interpretable Machine Learning with Neural Netscited by 319
  20. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  21. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  22. 2019Entity, Relation, and Event Extraction with Contextualized Span Representationscited by 262
  23. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  24. 2019Visualizing and Measuring the Geometry of BERTcited by 216
  25. 2015Visualizing and Understanding Neural Models in NLPcited by 200
  26. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 191
  27. 2015Visualizing and Understanding Recurrent Networkscited by 160
  28. 2018Visual Interpretability for Deep Learning: a Surveycited by 155
  29. 2019The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectivescited by 139
  30. 2020Self-Attention Attribution: Interpreting Information Interactions Inside Transformercited by 125
  31. 2018Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontierscited by 117
  32. 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 108
  33. 2020Unifying Graph Convolutional Neural Networks and Label Propagationcited by 98
  34. 2019The emergence of number and syntax units in LSTM language modelscited by 94
  35. 2019FewRel 2.0: Towards More Challenging Few-Shot Relation Classificationcited by 94
  36. 2020Behind the Scene: Revealing the Secrets of Pre-trained Vision-and-Language Modelscited by 90
  37. 2020BERTology Meets Biology: Interpreting Attention in Protein Language Modelscited by 85
  38. 2020OpenAttack: An Open-source Textual Adversarial Attack Toolkitcited by 84

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 2.

4 papers.

  1. 2023An Interdisciplinary Outlook on Large Language Models for Scientific Researchcited by 15
  2. 2025Explainability in Practice: A Survey of Explainable NLP Across Various Domainscited by 11
  3. 2022ProVe: A Pipeline for Automated Provenance Verification of Knowledge Graphs against Textual Sourcescited by 5
  4. 2025ExpliCIT-QA: Explainable Code-Based Image Table Question Answeringcited by 1