# Active Learning for Visual Question Answering: An Empirical Study

arXiv: https://arxiv.org/abs/1711.01732 · HTML: https://mlsanity.com/paper/1711.01732/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2017.

## Built on by (most cited first, 11 of 11)

- [Replay in Deep Learning: Current Approaches and Missing Biological Elements](https://mlsanity.com/paper/2104.04132/) (2021)
- [Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering](https://mlsanity.com/paper/2107.02331/) (2021)
- [Single-Modal Entropy based Active Learning for Visual Question Answering](https://mlsanity.com/paper/2110.10906/) (2021)
- [Combining Data Generation and Active Learning for Low-Resource Question Answering](https://mlsanity.com/paper/2211.14880/) (2022)
- [Can I see an Example? Active Learning the Long Tail of Attributes and Relations](https://mlsanity.com/paper/2203.06215/) (2022)
- [Optimizing Multi-Domain Performance with Active Learning-based Improvement Strategies](https://mlsanity.com/paper/2304.06277/) (2023)
- [AL-iGAN: An Active Learning Framework for Tunnel Geological Reconstruction Based on TBM Operational Data](https://mlsanity.com/paper/2212.00965/) (2022)
- [Perturbation-based Active Learning for Question Answering](https://mlsanity.com/paper/2311.02345/) (2023)
- [Maximally Separated Active Learning](https://mlsanity.com/paper/2411.17444/) (2024)
- [Active Learning Methods for Efficient Data Utilization and Model Performance Enhancement](https://mlsanity.com/paper/2504.16136/) (2025)
- [Cost-efficient Active Learning for Referring Image Segmentation and Grounding](https://mlsanity.com/paper/2608.30621/) (2026)

## Abstract (arXiv)

We present an empirical study of active learning for Visual Question Answering, where a deep VQA model selects informative question-image pairs from a pool and queries an oracle for answers to maximally improve its performance under a limited query budget. Drawing analogies from human learning, we explore cramming (entropy), curiosity-driven (expected model change), and goal-driven (expected error reduction) active learning approaches, and propose a fast and effective goal-driven active learning scoring function to pick question-image pairs for deep VQA models under the Bayesian Neural Network framework. We find that deep VQA models need large amounts of training data before they can start asking informative questions. But once they do, all three approaches outperform the random selection baseline and achieve significant query savings. For the scenario where the model is allowed to ask generic questions about images but is evaluated only on specific questions (e.g., questions whose answer is either yes or no), our proposed goal-driven scoring function performs the best.