Data to 7 Oct 2026

Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers

arXiv 2103.15679 ↗First posted 29 Mar 2021cs.CV cs.LGcited by 316

Not on the map. Why?

Cited by 75 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Transformers are increasingly dominating multi-modal reasoning tasks, such as visual question answering, achieving state-of-the-art results thanks to their ability to contextualize information using the self-attention and co-attention mechanisms. These attention modules also play a role in other computer vision tasks including object detection and image segmentation. Unlike Transformers that only use self-attention, Transformers with co-attention require to consider multiple attention maps in parallel in order to highlight the information that is relevant to the prediction in the model's input. In this work, we propose the first method to explain prediction by any Transformer-based architecture, including bi-modal Transformers and Transformers with co-attentions. We provide generic solutions and apply these to the three most commonly used of these architectures: (i) pure self-attention, (ii) self-attention combined with co-attention, and (iii) encoder-decoder attention. We show that our method is superior to all existing methods which are adapted from single modality explainability.

Key facts. Cited by 316 papers in the map (data through 7 Oct 2026). Builds on 44 of its 51 references that are in the map. First posted on arXiv 29 Mar 2021.

Builds on

44 of 51 references.

References · 44

2017Attention Is All You Needcited by 59,932
2015Deep Residual Learning for Image Recognitioncited by 51,809
2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
2014Microsoft COCO: Common Objects in Contextcited by 17,498
2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
2020End-to-End Object Detection with Transformerscited by 7,384
2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
2017A Unified Approach to Interpreting Model Predictionscited by 6,488
2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
2020Training data-efficient image transformers & distillation through attentioncited by 4,560

32 more

2021Zero-Shot Text-to-Image Generationcited by 4,276
2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
2017Axiomatic Attribution for Deep Networkscited by 3,350
2015VQA: Visual Question Answeringcited by 2,489
2015Learning Deep Features for Discriminative Localizationcited by 2,430
2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
2013Visualizing and Understanding Convolutional Networkscited by 2,278
2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
2017Learning Important Features Through Propagating Activation Differencescited by 1,454
2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
2017SmoothGrad: removing noise by adding noisecited by 1,097
2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
2020Quantifying Attention Flow in Transformerscited by 759
2020Transformer Interpretability Beyond Attention Visualizationcited by 620
2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
2020End-to-End Video Instance Segmentation with Transformerscited by 443
2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
2016Top-down Neural Attention by Excitation Backpropcited by 318
2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 257
2017Real Time Image Saliency for Black Box Classifierscited by 246
2019Full-Gradient Representation for Neural Network Visualizationcited by 189
2017Interpreting Deep Visual Representations via Network Dissectioncited by 103
2018L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Datacited by 100
2015Visualizing Deep Convolutional Neural Networks Using Natural Pre-Imagescited by 93
2019Relative Attributing Propagation: Interpreting the Comparative Contributions of Individual Units in Deep Neural Networkscited by 49
2018Understanding Individual Decisions of CNNs via Contrastive Backpropagationcited by 40
2019Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagationcited by 29
2021Local Memory Attention for Fast Video Semantic Segmentationcited by 25
2020Visualization of Supervised and Self-Supervised Neural Networks via Attribution Guided Factorizationcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 41; 2023: 65; 2024: 81; 2025: 72; 2026: 53.

316 papers.

Sort:

By area:

2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
2023LERF: Language Embedded Radiance Fieldscited by 538
2022Multimodal Learning with Transformers: A Surveycited by 399
2022Visual Classification via Description from Large Language Modelscited by 331
2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 311
2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
2023Break-A-Scene: Extracting Multiple Concepts from a Single Imagecited by 230
2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
2023Teaching CLIP to Count to Tencited by 194
2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lenscited by 139
2022ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehensioncited by 119

38 more

2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 116
2023Iterative Prompt Learning for Unsupervised Backlit Image Enhancementcited by 107
2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 106
2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 103
2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
2021Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understandingcited by 97
2022Semantic Abstraction: Open-World 3D Scene Understanding from 2D Vision-Language Modelscited by 95
2023A Closer Look at the Explainability of Contrastive Language-Image Pre-trainingcited by 95
2023Language Embedded Radiance Fields for Zero-Shot Task-Oriented Graspingcited by 91
2024The Hidden Attention of Mamba Modelscited by 88
2024DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Modelscited by 74
2024DreamLIP: Language-Image Pre-training with Long Captionscited by 72
2023X-Pruner: eXplainable Pruning for Vision Transformerscited by 66
2024What's in the Image? A Deep-Dive into the Vision of Vision Language Modelscited by 66
2024Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Modelscited by 65
2024Cross-modal Information Flow in Multimodal Large Language Modelscited by 65
2022Measuring the Mixing of Contextual Information in the Transformercited by 63
2022CLIPascene: Scene Sketching with Different Types and Levels of Abstractioncited by 63
2022Composition-aware Graphic Layout GAN for Visual-textual Presentation Designscited by 59
2022CLIPasso: Semantically-Aware Object Sketchingcited by 57
2023PIGEON: Predicting Image Geolocationscited by 55
2022VL-InterpreT: An Interactive Visualization Tool for Interpreting Vision-Language Transformerscited by 53
2022Language Modelling with Pixelscited by 53
2024Visual Agents as Fast and Slow Thinkerscited by 53
2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Modelscited by 50
2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
2022Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformercited by 45
2023Data Roaming and Quality Assessment for Composed Image Retrievalcited by 45
2023FLIP: Cross-domain Face Anti-spoofing with Language Guidancecited by 45
2022Rethinking Attention-Model Explainability through Faithfulness Violation Testcited by 43
2022MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Taskscited by 42
2022Optimizing Relevance Maps of Vision Transformers Improves Robustnesscited by 41
2023If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selectioncited by 40
2024CFPL-FAS: Class Free Prompt Learning for Generalizable Face Anti-spoofingcited by 39
2024Where am I? Cross-View Geo-localization with Natural Language Descriptionscited by 38
2024Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detectioncited by 35
2025Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMscited by 35

50 most cited of 316 All 316 (JSON)