# Overcoming Language Priors for Visual Question Answering Based on Knowledge Distillation

arXiv: https://arxiv.org/abs/2501.05690 · HTML: https://mlsanity.com/paper/2501.05690/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- Builds on 30 of its 43 references that are in the map.
- First posted on arXiv 10 Jan 2025.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 30 of 30)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [Stacked Attention Networks for Image Question Answering](https://mlsanity.com/paper/1511.02274/) (2015)
- [Counterfactual VQA: A Cause-Effect Look at Language Bias](https://mlsanity.com/paper/2006.04315/) (2020)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Counterfactual Samples Synthesizing for Robust Visual Question Answering](https://mlsanity.com/paper/2003.06576/) (2020)
- [Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded](https://mlsanity.com/paper/1902.03751/) (2019)
- [Overcoming Language Priors in Visual Question Answering with Adversarial Regularization](https://mlsanity.com/paper/1810.03649/) (2018)
- [MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering](https://mlsanity.com/paper/2009.08566/) (2020)
- [Self-Critical Reasoning for Robust Visual Question Answering](https://mlsanity.com/paper/1905.09998/) (2019)
- [Is Label Smoothing Truly Incompatible with Knowledge Distillation: An Empirical Study](https://mlsanity.com/paper/2104.00676/) (2021)
- [Overcoming Language Priors with Self-supervised Learning for Visual Question Answering](https://mlsanity.com/paper/2012.11528/) (2020)
- [Loss re-scaling VQA: Revisiting the LanguagePrior Problem from a Class-imbalance View](https://mlsanity.com/paper/2010.16010/) (2020)
- [Greedy Gradient Ensemble for Robust Visual Question Answering](https://mlsanity.com/paper/2107.12651/) (2021)
- [Introspective Distillation for Robust Question Answering](https://mlsanity.com/paper/2111.01026/) (2021)
- [Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering](https://mlsanity.com/paper/2110.01013/) (2021)
- [AdaVQA: Overcoming Language Priors with Adapted Margin Cosine Loss](https://mlsanity.com/paper/2105.01993/) (2021)
- [Rethinking Data Augmentation for Robust Visual Question Answering](https://mlsanity.com/paper/2207.08739/) (2022)
- [Discovering the Unknown Knowns: Turning Implicit Knowledge in the Dataset into Explicit Training Examples for Visual Question Answering](https://mlsanity.com/paper/2109.06122/) (2021)
- [Generative Bias for Robust Visual Question Answering](https://mlsanity.com/paper/2208.00690/) (2022)
- [Declaration-based Prompt Tuning for Visual Question Answering](https://mlsanity.com/paper/2205.02456/) (2022)
- [Learning content and context with language bias for Visual Question Answering](https://mlsanity.com/paper/2012.11134/) (2020)
- [General Greedy De-bias Learning](https://mlsanity.com/paper/2112.10572/) (2021)
- [An Empirical Study on the Language Modal in Visual Question Answering](https://mlsanity.com/paper/2305.10143/) (2023)
- [Detection-based Intermediate Supervision for Visual Question Answering](https://mlsanity.com/paper/2312.16012/) (2023)

## Built on by (most cited first, 1 of 1)

- [Multi-Sourced Compositional Generalization in Visual Question Answering](https://mlsanity.com/paper/2505.23045/) (2025)

## Abstract (arXiv)

Previous studies have pointed out that visual question answering (VQA) models are prone to relying on language priors for answer predictions. In this context, predictions often depend on linguistic shortcuts rather than a comprehensive grasp of multimodal knowledge, which diminishes their generalization ability. In this paper, we propose a novel method, namely, KDAR, leveraging knowledge distillation to address the prior-dependency dilemmas within the VQA task. Specifically, the regularization effect facilitated by soft labels from a well-trained teacher is employed to penalize overfitting to the most common answers. The soft labels, which serve a regularization role, also provide semantic guidance that narrows the range of candidate answers. Additionally, we design an adaptive sample-wise reweighting learning strategy to further mitigate bias by dynamically adjusting the importance of each sample. Experimental results demonstrate that our method enhances performance in both OOD and IID settings. Our method achieves state-of-the-art performance on the VQA-CPv2 out-of-distribution (OOD) benchmark, significantly outperforming previous state-of-the-art approaches.