mlsanityData to 7 Oct 2026

Single-Modal Entropy based Active Learning for Visual Question Answering

arXiv 2110.10906 ↗First posted 21 Oct 2021cs.CV cs.AI cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Constructing a large-scale labeled dataset in the real world, especially for high-level tasks (eg, Visual Question Answering), can be expensive and time-consuming. In addition, with the ever-growing amounts of data and architecture complexity, Active Learning has become an important aspect of computer vision research. In this work, we address Active Learning in the multi-modal setting of Visual Question Answering (VQA). In light of the multi-modal inputs, image and question, we propose a novel method for effective sample acquisition through the use of ad hoc single-modal branches for each input to leverage its information. Our mutual information based sample acquisition strategy Single-Modal Entropic Measure (SMEM) in addition to our self-distillation technique enables the sample acquisitor to exploit all present modalities and find the most informative samples. Our novel idea is simple to implement, cost-efficient, and readily adaptable to other multi-modal tasks. We confirm our findings on various VQA datasets through state-of-the-art performance by comparing to existing Active Learning baselines.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 41 of its 62 references that are in the map. First posted on arXiv 21 Oct 2021.

Builds on

41 of 62 references.

References · 41

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2016Learning without Forgettingcited by 2,465
  7. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  9. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  10. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  11. 2017Deep Bayesian Active Learning with Image Datacited by 812
  12. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
29 more
  1. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
  2. 2019Learning Loss for Active Learningcited by 335
  3. 2015Visual7W: Grounded Question Answering in Imagescited by 315
  4. 2019Variational Adversarial Active Learningcited by 284
  5. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  6. 2018Bilinear Attention Networkscited by 277
  7. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  8. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
  9. 2019RUBi: Reducing Unimodal Biases in Visual Question Answeringcited by 183
  10. 2015DenseCap: Fully Convolutional Localization Networks for Dense Captioningcited by 176
  11. 2017Deep Active Learning for Named Entity Recognitioncited by 108
  12. 2018Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregationcited by 106
  13. 2020Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learningcited by 105
  14. 2020Detecting Human-Object Interactions with Action Co-occurrence Priorscited by 72
  15. 2020State-Relabeling Adversarial Active Learningcited by 60
  16. 2019Just Ask:An Interactive Learning Framework for Vision and Language Navigationcited by 53
  17. 2018Show, Tell and Discriminate: Image Captioning by Self-retrieval with Partially Labeled Datacited by 37
  18. 2019Dense Relational Captioning: Triple-Stream Networks for Relationship-Based Captioningcited by 35
  19. 2021LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentationcited by 33
  20. 2019Image Captioning with Very Scarce Supervised Data: Adversarial Semi-Supervised Learning Approachcited by 30
  21. 2017Learning by Asking Questionscited by 25
  22. 2021MCDAL: Maximum Classifier Discrepancy for Active Learningcited by 18
  23. 2018Disjoint Multi-task Learning between Heterogeneous Human-centric Taskscited by 16
  24. 2021Dealing with Missing Modalities in the Visual Question Answer-Difference Prediction Task through Knowledge Distillationcited by 14
  25. 2017Active Learning for Visual Question Answering: An Empirical Studycited by 11
  26. 2018Learning to Caption Images through a Lifetime by Asking Questionscited by 11
  27. 2020Dense Relational Image Captioning via Multi-task Triple-Stream Networkscited by 11
  28. 2021ACP++: Action Co-occurrence Priors for Human-Object Interaction Detectioncited by 11
  29. 2017Visual Question Answering as a Meta Learning Taskcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 1; 2026: 1.

6 papers.

  1. 2024Preserving Multi-Modal Capabilities of Pre-trained VLMs for Improving Vision-Linguistic Compositionalitycited by 17
  2. 2022Generative Bias for Robust Visual Question Answeringcited by 12
  3. 2024Let Me Finish My Sentence: Video Temporal Grounding with Holistic Text Understandingcited by 7
  4. 2023Semi-Supervised Image Captioning by Adversarially Propagating Labeled Datacited by 4
  5. 2025Draft and Refine with Visual Expertscited by 1
  6. 2026Cost-efficient Active Learning for Referring Image Segmentation and Groundingcited by 0