mlsanityData to 7 Oct 2026

T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP

arXiv 2108.13587 ↗First posted 31 Aug 2021cs.CL cs.HCcited by 11

Not on the map. Why?

Abstract (arXiv)

Transformers are the dominant architecture in NLP, but their training and fine-tuning is still very challenging. In this paper, we present the design and implementation of a visual analytic framework for assisting researchers in such process, by providing them with valuable insights about the model's intrinsic properties and behaviours. Our framework offers an intuitive overview that allows the user to explore different facets of the model (e.g., hidden states, attention) through interactive visualization, and allows a suite of built-in algorithms that compute the importance of model components and different parts of the input sequence. Case studies and feedback from a user focus group indicate that the framework is useful, and suggest several improvements.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 34 of its 44 references that are in the map. First posted on arXiv 31 Aug 2021.

Builds on

34 of 44 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  6. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  8. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  9. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  10. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  11. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  12. 2015Teaching Machines to Read and Comprehendcited by 1,039
22 more
  1. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  2. 2019Attention is not Explanationcited by 916
  3. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  4. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 832
  5. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  6. 2019Attention is not not Explanationcited by 629
  7. 2019Text Summarization with Pretrained Encoderscited by 553
  8. 2019Importance Estimation for Neural Network Pruningcited by 531
  9. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  10. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  11. 2019Revealing the Dark Secrets of BERTcited by 300
  12. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  13. 2016Understanding Neural Networks through Representation Erasurecited by 223
  14. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  15. 2018Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontierscited by 117
  16. 2020What Happens To BERT Embeddings During Fine-tuning?cited by 113
  17. 2019AllenNLP Interpret: A Framework for Explaining Predictions of NLP Modelscited by 74
  18. 2019Visualizing and Understanding the Effectiveness of BERTcited by 67
  19. 2017ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Modelscited by 56
  20. 2016Embedding Projector: Interactive Visualization and Interpretation of Embeddingscited by 43
  21. 2018DeepTracker: Visualizing the Training Process of Convolutional Neural Networkscited by 7
  22. 2020VisBERT: Hidden-State Visualizations for Transformerscited by 6

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 5; 2024: 1; 2025: 2.

11 papers.

  1. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
  2. 2022VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformerscited by 53
  3. 2023Foundation Models Meet Visualizations: Challenges and Opportunitiescited by 31
  4. 2023How Does Attention Work in Vision Transformers? A Visual Analytics Attemptcited by 29
  5. 2023BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Modelscited by 8
  6. 2021Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentationcited by 6
  7. 2025Charts-of-Thought: Enhancing LLM Visualization Literacy Through Structured Data Extractioncited by 6
  8. 2023Finspector: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Modelscited by 5
  9. 2024POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Modelscited by 1
  10. 2023XAI for In-hospital Mortality Prediction via Multimodal ICU Datacited by 0
  11. 2025Visual Attention Exploration in Vision-Based Mamba Modelscited by 0