mlsanityData to 14 Oct 2026

LPF: A Language-Prior Feedback Objective Function for De-biased Visual Question Answering

arXiv 2105.14300 ↗First posted 29 May 2021cs.CV cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Most existing Visual Question Answering (VQA) systems tend to overly rely on language bias and hence fail to reason from the visual clue. To address this issue, we propose a novel Language-Prior Feedback (LPF) objective function, to re-balance the proportion of each answer's loss value in the total VQA loss. The LPF firstly calculates a modulating factor to determine the language bias using a question-only branch. Then, the LPF assigns a self-adaptive weight to each training sample in the training process. With this reweighting mechanism, the LPF ensures that the total VQA loss can be reshaped to a more balanced form. By this means, the samples that require certain visual information to predict will be efficiently used during training. Our method is simple to implement, model-agnostic, and end-to-end trainable. We conduct extensive experiments and the results show that the LPF (1) brings a significant improvement over various VQA models, (2) achieves competitive performance on the bias-sensitive VQA-CP v2 benchmark.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 29 May 2021.

Builds on

33 of 55 references.

References · 33

  1. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  2. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  3. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  4. 2015VQA: Visual Question Answeringcited by 2,506
  5. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  6. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,312
  7. 2017Focal Loss for Dense Object Detectioncited by 752
  8. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 339
  9. 2018Bilinear Attention Networkscited by 280
  10. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 280
  11. 2020Counterfactual VQA: A Cause-Effect Look at Language Biascited by 266
  12. 2019RUBi: Reducing Unimodal Biases in Visual Question Answeringcited by 185
21 more
  1. 2020Counterfactual Samples Synthesizing for Robust Visual Question Answeringcited by 153
  2. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 128
  3. 2019End-to-End Bias Mitigation by Modelling Biases in Corporacited by 128
  4. 2018Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answeringcited by 114
  5. 2016Human Attention in Visual Question Answering: Do Humans and Deep Networks Look at the Same Regions?cited by 111
  6. 2018Overcoming Language Priors in Visual Question Answering with Adversarial Regularizationcited by 105
  7. 2019Cycle-Consistency for Robust Visual Question Answeringcited by 105
  8. 2020On the Value of Out-of-Distribution Testing: An Example of Goodhart's Lawcited by 74
  9. 2019MUREL: Multimodal Relational Reasoning for Visual Question Answeringcited by 68
  10. 2020Learning What Makes a Difference from Counterfactual Examples and Gradient Supervisioncited by 66
  11. 2018Pythia v0.1: the Winning Entry to the VQA Challenge 2018cited by 62
  12. 2019BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and Visual Relationship Detectioncited by 62
  13. 2019Self-Critical Reasoning for Robust Visual Question Answeringcited by 61
  14. 2018Explainable Neural Computation via Stack Neural Module Networkscited by 54
  15. 2020Overcoming Language Priors with Self-supervised Learning for Visual Question Answeringcited by 43
  16. 2020Reducing Language Biases in Visual Question Answering with Visually-Grounded Question Encodercited by 34
  17. 2019Adversarial Regularization for Visual Question Answering: Strengths, Shortcomings, and Side Effectscited by 25
  18. 2020Visual Grounding Methods for VQA are Working for the Wrong Reasons!cited by 18
  19. 2019Quantifying and Alleviating the Language Prior Problem in Visual Question Answeringcited by 13
  20. 2019Generating Question Relevant Captions to Aid Visual Question Answeringcited by 10
  21. 2019Citation concept analysis (CCA) - A new form of citation analysis revealing the usefulness of concepts for other researchers illustrated by two exemplary case studies including classic books by Thomas S. Kuhn and Karl R. Poppercited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 4; 2024: 1; 2025: 4; 2026: 1.

17 papers.

Sort:

By area:

  1. 2022Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQAcited by 30
  2. 2025Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcutscited by 29
  3. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  4. 2023Visual Question Answering: A Survey on Techniques and Common Trends in Recent Literaturecited by 21
  5. 2022Rethinking Data Augmentation for Robust Visual Question Answeringcited by 18
  6. 2022Visual Perturbation-aware Collaborative Learning for Overcoming the Language Prior Problemcited by 18
  7. 2022Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learningcited by 9
  8. 2021Language bias in Visual Question Answering: A Survey and Taxonomycited by 6
  9. 2021Achieving Human Parity on Visual Question Answeringcited by 5
  10. 2026Watch Before You Answer: Learning from Visually Grounded Post-Trainingcited by 5
  11. 2023An Empirical Study on the Language Modal in Visual Question Answeringcited by 3
  12. 2024Eliminating the Language Bias for Visual Question Answering with fine-grained Causal Interventioncited by 3
5 more
  1. 2023Artificial-Spiking Hierarchical Networks for Vision-Language Representation Learningcited by 1
  2. 2025MiMu: Mitigating Multiple Shortcut Learning Behavior of Transformerscited by 1
  3. 2025Cause-Effect Driven Optimization for Robust Medical Visual Question Answering with Language Biasescited by 1
  4. 2022Compressing And Debiasing Vision-Language Pre-Trained Models for Visual Question Answeringcited by 0
  5. 2025Improving vision-language alignment with graph spiking hybrid Networkscited by 0