# MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering

arXiv: https://arxiv.org/abs/2009.08566 · HTML: https://mlsanity.com/paper/2009.08566/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2020.

## Built on by (most cited first, 30 of 63)

- [VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena](https://mlsanity.com/paper/2112.07566/) (2021)
- [Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models](https://mlsanity.com/paper/2310.05338/) (2023)
- [Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering](https://mlsanity.com/paper/2104.03149/) (2021)
- [Weakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering](https://mlsanity.com/paper/2109.04014/) (2021)
- [SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answering](https://mlsanity.com/paper/2204.02285/) (2022)
- [Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA](https://mlsanity.com/paper/2210.04692/) (2022)
- [ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models](https://mlsanity.com/paper/2311.07022/) (2023)
- [Introspective Distillation for Robust Question Answering](https://mlsanity.com/paper/2111.01026/) (2021)
- [Robust Visual Question Answering: Datasets, Methods, and Future Challenges](https://mlsanity.com/paper/2307.11471/) (2023)
- [Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering](https://mlsanity.com/paper/2110.01013/) (2021)
- [Visual Question Answering: A Survey on Techniques and Common Trends in Recent Literature](https://mlsanity.com/paper/2305.11033/) (2023)
- [Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization](https://mlsanity.com/paper/2205.12191/) (2022)
- [VisQA: X-raying Vision and Language Reasoning in Transformers](https://mlsanity.com/paper/2104.00926/) (2021)
- [Check It Again: Progressive Visual Question Answering via Visual Entailment](https://mlsanity.com/paper/2106.04605/) (2021)
- [Rethinking Data Augmentation for Robust Visual Question Answering](https://mlsanity.com/paper/2207.08739/) (2022)
- [CPL: Counterfactual Prompt Learning for Vision and Language Models](https://mlsanity.com/paper/2210.10362/) (2022)
- [Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering](https://mlsanity.com/paper/2109.06122/) (2021)
- [Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization](https://mlsanity.com/paper/2503.17928/) (2025)
- [Understanding Finetuning for Factual Knowledge Extraction from Language Models](https://mlsanity.com/paper/2301.11293/) (2023)
- [Generative Bias for Robust Visual Question Answering](https://mlsanity.com/paper/2208.00690/) (2022)
- [Domain-robust VQA with diverse datasets and methods but no target labels](https://mlsanity.com/paper/2103.15974/) (2021)
- [A Closer Look at Debiased Temporal Sentence Grounding in Videos: Dataset, Metric, and Approach](https://mlsanity.com/paper/2203.05243/) (2022)
- [X-GGM: Graph Generative Modeling for Out-of-Distribution Generalization in Visual Question Answering](https://mlsanity.com/paper/2107.11576/) (2021)
- [Semantically Distributed Robust Optimization for Vision-and-Language Inference](https://mlsanity.com/paper/2110.07165/) (2021)
- [Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning](https://mlsanity.com/paper/2210.04563/) (2022)
- [Uncovering the Full Potential of Visual Grounding Methods in VQA](https://mlsanity.com/paper/2401.07803/) (2024)
- [SimVQA: Exploring Simulated Environments for Visual Question Answering](https://mlsanity.com/paper/2203.17219/) (2022)
- [A Comprehensive Survey on Visual Question Answering Datasets and Algorithms](https://mlsanity.com/paper/2411.11150/) (2024)
- [Cross-Modal Generative Augmentation for Visual Question Answering](https://mlsanity.com/paper/2105.04780/) (2021)
- [Dataset Bias Mitigation in Multiple-Choice Visual Question Answering and Beyond](https://mlsanity.com/paper/2310.14670/) (2023)

## Abstract (arXiv)

While progress has been made on the visual question answering leaderboards, models often utilize spurious correlations and priors in datasets under the i.i.d. setting. As such, evaluation on out-of-distribution (OOD) test samples has emerged as a proxy for generalization. In this paper, we present MUTANT, a training paradigm that exposes the model to perceptually similar, yet semantically distinct mutations of the input, to improve OOD generalization, such as the VQA-CP challenge. Under this paradigm, models utilize a consistency-constrained training objective to understand the effect of semantic changes in input (question-image pair) on the output (answer). Unlike existing methods on VQA-CP, MUTANT does not rely on the knowledge about the nature of train and test answer distributions. MUTANT establishes a new state-of-the-art accuracy on VQA-CP with a 10.57% improvement. Our work opens up avenues for the use of semantic input mutations for OOD generalization in question answering.