# Language bias in Visual Question Answering: A Survey and Taxonomy

arXiv: https://arxiv.org/abs/2111.08531 · HTML: https://mlsanity.com/paper/2111.08531/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 109 references that are in the map.
- First posted on arXiv 16 Nov 2021.

## Builds on (most cited first, 30 of 51)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [Long-tail learning via logit adjustment](https://mlsanity.com/paper/2007.07314/) (2020)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Stacked Attention Networks for Image Question Answering](https://mlsanity.com/paper/1511.02274/) (2015)
- [Counterfactual VQA: A Cause-Effect Look at Language Bias](https://mlsanity.com/paper/2006.04315/) (2020)
- [DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/1908.11540/) (2019)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Counterfactual Samples Synthesizing for Robust Visual Question Answering](https://mlsanity.com/paper/2003.06576/) (2020)
- [Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded](https://mlsanity.com/paper/1902.03751/) (2019)
- [Relation-Aware Graph Attention Network for Visual Question Answering](https://mlsanity.com/paper/1903.12314/) (2019)
- [Learning by Abstraction: The Neural State Machine](https://mlsanity.com/paper/1907.03950/) (2019)
- [Overcoming Language Priors in Visual Question Answering with Adversarial Regularization](https://mlsanity.com/paper/1810.03649/) (2018)
- [On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law](https://mlsanity.com/paper/2005.09241/) (2020)
- [MUREL: Multimodal Relational Reasoning for Visual Question Answering](https://mlsanity.com/paper/1902.09487/) (2019)
- [Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision](https://mlsanity.com/paper/2004.09034/) (2020)
- [MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering](https://mlsanity.com/paper/2009.08566/) (2020)
- [Self-Critical Reasoning for Robust Visual Question Answering](https://mlsanity.com/paper/1905.09998/) (2019)
- [Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies](https://mlsanity.com/paper/2010.10802/) (2020)

## Built on by (most cited first, 6 of 6)

- [From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models](https://mlsanity.com/paper/2212.10846/) (2022)
- [All You May Need for VQA are Image Captions](https://mlsanity.com/paper/2205.01883/) (2022)
- [From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities](https://mlsanity.com/paper/2311.00308/) (2023)
- [CIC: A Framework for Culturally-Aware Image Captioning](https://mlsanity.com/paper/2402.05374/) (2024)
- [Understanding Museum Exhibits using Vision-Language Reasoning](https://mlsanity.com/paper/2412.01370/) (2024)
- [Overcoming Language Bias in Remote Sensing Visual Question Answering via Adversarial Training](https://mlsanity.com/paper/2306.00483/) (2023)

## Abstract (arXiv)

Visual question answering (VQA) is a challenging task, which has attracted more and more attention in the field of computer vision and natural language processing. However, the current visual question answering has the problem of language bias, which reduces the robustness of the model and has an adverse impact on the practical application of visual question answering. In this paper, we conduct a comprehensive review and analysis of this field for the first time, and classify the existing methods according to three categories, including enhancing visual information, weakening language priors, data enhancement and training strategies. At the same time, the relevant representative methods are introduced, summarized and analyzed in turn. The causes of language bias are revealed and classified. Secondly, this paper introduces the datasets mainly used for testing, and reports the experimental results of various existing methods. Finally, we discuss the possible future research directions in this field.