# QIRL: Optimized Question-Image Relation Learning for Bias-Robust Visual Question Answering

arXiv: https://arxiv.org/abs/2504.03337 · HTML: https://mlsanity.com/paper/2504.03337/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 4 Apr 2025.

## Builds on (most cited first, 27 of 27)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Stacked Cross Attention for Image-Text Matching](https://mlsanity.com/paper/1803.08024/) (2018)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA](https://mlsanity.com/paper/2109.05014/) (2021)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Stacked Attention Networks for Image Question Answering](https://mlsanity.com/paper/1511.02274/) (2015)
- [Counterfactual Samples Synthesizing for Robust Visual Question Answering](https://mlsanity.com/paper/2003.06576/) (2020)
- [Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded](https://mlsanity.com/paper/1902.03751/) (2019)
- [KAT: A Knowledge Augmented Transformer for Vision-and-Language](https://mlsanity.com/paper/2112.08614/) (2021)
- [Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone](https://mlsanity.com/paper/2206.07643/) (2022)
- [Overcoming Language Priors in Visual Question Answering with Adversarial Regularization](https://mlsanity.com/paper/1810.03649/) (2018)
- [PromptCap: Prompt-Guided Task-Aware Image Captioning](https://mlsanity.com/paper/2211.09699/) (2022)
- [CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling](https://mlsanity.com/paper/1811.10996/) (2018)
- [Self-Critical Reasoning for Robust Visual Question Answering](https://mlsanity.com/paper/1905.09998/) (2019)
- [REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering](https://mlsanity.com/paper/2206.01201/) (2022)
- [Overcoming Language Priors with Self-supervised Learning for Visual Question Answering](https://mlsanity.com/paper/2012.11528/) (2020)
- [Loss re-scaling VQA: Revisiting the LanguagePrior Problem from a Class-imbalance View](https://mlsanity.com/paper/2010.16010/) (2020)
- [Unshuffling Data for Improved Generalization](https://mlsanity.com/paper/2002.11894/) (2020)
- [Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering](https://mlsanity.com/paper/2110.01013/) (2021)
- [Refined Semantic Enhancement towards Frequency Diffusion for Video Captioning](https://mlsanity.com/paper/2211.15076/) (2022)
- [Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization](https://mlsanity.com/paper/2503.17928/) (2025)
- [Answer Questions with Right Image Regions: A Visual Attention Regularization Approach](https://mlsanity.com/paper/2102.01916/) (2021)

## Abstract (arXiv)

Existing bias mitigation methods for Visual Question Answering (VQA), a typical Artificial intelligence application, endure two main limitations. First, they fail to capture the optimal relation between images and texts, as prevailing learning frameworks lack the capacity to extract deep correlations from highly contrasting samples. Second, they overlook assessing Question-Image (QI) relevance during inference, since prior work has not examined the degree of input relevance in debiasing studies. To address these issues, we propose a novel neural network framework termed Optimized Question-Image Relation Learning (QIRL), which provides a reliable implementation of artificial intelligence for VQA tasks and improves the robustness of conventional VQA models through a generation-driven self-supervised learning strategy. Specifically, two modules are introduced. The Negative Image Generation (NIG) module automatically produces highly irrelevant QI pairs during training to strengthen relational learning. In contrast, the Irrelevant Sample Identification (ISI) module enhances model robustness by detecting and filtering out irrelevant inputs, thereby reducing prediction errors. Moreover, to verify the effectiveness of filtering out unrelated QI pairs in mitigating output errors, we propose a specialized metric to evaluate the ISI module's performance. Notably, our approach is model-agnostic and can be seamlessly integrated with various VQA architectures. Extensive experiments on VQA-CPv2 and VQA-v2 datasets demonstrate the effectiveness and generalization ability of our method. Our approach achieves state-of-the-art performance.