# VisQA: X-raying Vision and Language Reasoning in Transformers

arXiv: https://arxiv.org/abs/2104.00926 · HTML: https://mlsanity.com/paper/2104.00926/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 68 references that are in the map.
- First posted on arXiv 2 Apr 2021.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [Deep Modular Co-Attention Networks for Visual Question Answering](https://mlsanity.com/paper/1906.10770/) (2019)
- [Large-Scale Adversarial Training for Vision-and-Language Representation Learning](https://mlsanity.com/paper/2006.06195/) (2020)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [Exploring Self-attention for Image Recognition](https://mlsanity.com/paper/2004.13621/) (2020)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformers](https://mlsanity.com/paper/2004.00849/) (2020)
- [Hierarchical Question-Image Co-Attention for Visual Question Answering](https://mlsanity.com/paper/1606.00061/) (2016)
- [Visualizing and Understanding Recurrent Networks](https://mlsanity.com/paper/1506.02078/) (2015)

## Built on by (most cited first, 19 of 19)

- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models](https://mlsanity.com/paper/2208.07852/) (2022)
- [AttentionViz: A Global View of Transformer Attention](https://mlsanity.com/paper/2305.03210/) (2023)
- [MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks](https://mlsanity.com/paper/2212.08158/) (2022)
- [3D Question Answering](https://mlsanity.com/paper/2112.08359/) (2021)
- [Neuron-Level Knowledge Attribution in Large Language Models](https://mlsanity.com/paper/2312.12141/) (2023)
- [How Does Attention Work in Vision Transformers? A Visual Analytics Attempt](https://mlsanity.com/paper/2303.13731/) (2023)
- [Visual Analytics For Machine Learning: A Data Perspective Survey](https://mlsanity.com/paper/2307.07712/) (2023)
- [Visual Question Answering: A Survey on Techniques and Common Trends in Recent Literature](https://mlsanity.com/paper/2305.11033/) (2023)
- [When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations](https://mlsanity.com/paper/2411.12701/) (2024)
- [Transformer Explainer: Learning LLM Transformers with Interactive Visual Explanation and Experimentation](https://mlsanity.com/paper/2408.04619/) (2024)
- [PromptExp: Multi-granularity Prompt Explanation of Large Language Models](https://mlsanity.com/paper/2410.13073/) (2024)
- [Visual Analytics for Efficient Image Exploration and User-Guided Image Captioning](https://mlsanity.com/paper/2311.01016/) (2023)
- [Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles](https://mlsanity.com/paper/2208.09665/) (2022)
- [POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models](https://mlsanity.com/paper/2406.03843/) (2024)
- [NEAT: Concept driven Neuron Attribution in LLMs](https://mlsanity.com/paper/2508.15875/) (2025)
- [VisMoDAl: Visual Analytics for Evaluating and Improving Corruption Robustness of Vision-Language Models](https://mlsanity.com/paper/2509.14571/) (2025)
- [On the Analogy between Human Brain and LLMs: Spotting Key Neurons in Grammar Perception](https://mlsanity.com/paper/2511.06519/) (2025)

## Abstract (arXiv)

Visual Question Answering systems target answering open-ended textual questions given input images. They are a testbed for learning high-level reasoning with a primary use in HCI, for instance assistance for the visually impaired. Recent research has shown that state-of-the-art models tend to produce answers exploiting biases and shortcuts in the training data, and sometimes do not even look at the input image, instead of performing the required reasoning steps. We present VisQA, a visual analytics tool that explores this question of reasoning vs. bias exploitation. It exposes the key element of state-of-the-art neural models -- attention maps in transformers. Our working hypothesis is that reasoning steps leading to model predictions are observable from attention distributions, which are particularly useful for visualization. The design process of VisQA was motivated by well-known bias examples from the fields of deep learning and vision-language reasoning and evaluated in two ways. First, as a result of a collaboration of three fields, machine learning, vision and language reasoning, and data analytics, the work lead to a better understanding of bias exploitation of neural models for VQA, which eventually resulted in an impact on its design and training through the proposition of a method for the transfer of reasoning patterns from an oracle model. Second, we also report on the design of VisQA, and a goal-oriented evaluation of VisQA targeting the analysis of a model decision process from multiple experts, providing evidence that it makes the inner workings of models accessible to users.