# Check It Again: Progressive Visual Question Answering via Visual Entailment

arXiv: https://arxiv.org/abs/2106.04605 · HTML: https://mlsanity.com/paper/2106.04605/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 28 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 23 of 23)

- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Visual Entailment: A Novel Task for Fine-Grained Image Understanding](https://mlsanity.com/paper/1901.06706/) (2019)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [DenseCap: Fully Convolutional Localization Networks for Dense Captioning](https://mlsanity.com/paper/1511.07571/) (2015)
- [Counterfactual Samples Synthesizing for Robust Visual Question Answering](https://mlsanity.com/paper/2003.06576/) (2020)
- [Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded](https://mlsanity.com/paper/1902.03751/) (2019)
- [Analyzing the Behavior of Visual Question Answering Models](https://mlsanity.com/paper/1606.07356/) (2016)
- [Yin and Yang: Balancing and Answering Binary Visual Questions](https://mlsanity.com/paper/1511.05099/) (2015)
- [Overcoming Language Priors in Visual Question Answering with Adversarial Regularization](https://mlsanity.com/paper/1810.03649/) (2018)
- [MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering](https://mlsanity.com/paper/2009.08566/) (2020)
- [Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference](https://mlsanity.com/paper/1907.04380/) (2019)
- [Overcoming Language Priors with Self-supervised Learning for Visual Question Answering](https://mlsanity.com/paper/2012.11528/) (2020)
- [Adversarial Regularization for Visual Question Answering: Strengths, Shortcomings, and Side Effects](https://mlsanity.com/paper/1906.08430/) (2019)
- [Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering](https://mlsanity.com/paper/1711.05116/) (2017)
- [RankQA: Neural Question Answering with Answer Re-Ranking](https://mlsanity.com/paper/1906.03008/) (2019)
- [Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension](https://mlsanity.com/paper/1805.06145/) (2018)

## Built on by (most cited first, 18 of 18)

- [Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA](https://mlsanity.com/paper/2210.04692/) (2022)
- [Robust Visual Question Answering: Datasets, Methods, and Future Challenges](https://mlsanity.com/paper/2307.11471/) (2023)
- [Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering](https://mlsanity.com/paper/2110.01013/) (2021)
- [Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering](https://mlsanity.com/paper/2404.12020/) (2024)
- [Generative Bias for Robust Visual Question Answering](https://mlsanity.com/paper/2208.00690/) (2022)
- [Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning](https://mlsanity.com/paper/2210.04563/) (2022)
- [General Greedy De-bias Learning](https://mlsanity.com/paper/2112.10572/) (2021)
- [Language bias in Visual Question Answering: A Survey and Taxonomy](https://mlsanity.com/paper/2111.08531/) (2021)
- [Understanding Social Media Cross-Modality Discourse in Linguistic Space](https://mlsanity.com/paper/2302.13311/) (2023)
- [Multimodal Reranking for Knowledge-Intensive Visual Question Answering](https://mlsanity.com/paper/2407.12277/) (2024)
- [Fine-Grained Visual Entailment](https://mlsanity.com/paper/2203.15704/) (2022)
- [Object Attribute Matters in Visual Question Answering](https://mlsanity.com/paper/2401.09442/) (2023)
- [Compressing And Debiasing Vision-Language Pre-Trained Models for Visual Question Answering](https://mlsanity.com/paper/2210.14558/) (2022)
- [SC-ML: Self-supervised Counterfactual Metric Learning for Debiased Visual Question Answering](https://mlsanity.com/paper/2304.01647/) (2023)
- [VEglue: Testing Visual Entailment Systems via Object-Aligned Joint Erasing](https://mlsanity.com/paper/2403.02581/) (2024)
- [Characterizing the Predictive Impact of Modalities with Supervised Latent-Variable Modeling](https://mlsanity.com/paper/2602.16979/) (2026)
- [Online Self-Calibration Against Hallucination in Vision-Language Models](https://mlsanity.com/paper/2605.00323/) (2026)
- [When to Rethink: Learning Multi-Perspective Self-Verification for Vision-Language Models](https://mlsanity.com/paper/2610.07018/) (2026)

## Abstract (arXiv)

While sophisticated Visual Question Answering models have achieved remarkable success, they tend to answer questions only according to superficial correlations between question and answer. Several recent approaches have been developed to address this language priors problem. However, most of them predict the correct answer according to one best output without checking the authenticity of answers. Besides, they only explore the interaction between image and question, ignoring the semantics of candidate answers. In this paper, we propose a select-and-rerank (SAR) progressive framework based on Visual Entailment. Specifically, we first select the candidate answers relevant to the question or the image, then we rerank the candidate answers by a visual entailment task, which verifies whether the image semantically entails the synthetic statement of the question and each candidate answer. Experimental results show the effectiveness of our proposed framework, which establishes a new state-of-the-art accuracy on VQA-CP v2 with a 7.55% improvement.