mlsanityData to 14 Oct 2026

Learning Sparse Mixture of Experts for Visual Question Answering

arXiv 1909.09192 ↗First posted 19 Sep 2019cs.LG cs.CL cs.CVcited by 2

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

There has been a rapid progress in the task of Visual Question Answering with improved model architectures. Unfortunately, these models are usually computationally intensive due to their sheer size which poses a serious challenge for deployment. We aim to tackle this issue for the specific task of Visual Question Answering (VQA). A Convolutional Neural Network (CNN) is an integral part of the visual processing pipeline of a VQA model (assuming the CNN is trained along with entire VQA model). In this project, we propose an efficient and modular neural architecture for the VQA task with focus on the CNN module. Our experiments demonstrate that a sparsely activated CNN based VQA model achieves comparable performance to a standard CNN based VQA model architecture.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 24 of its 33 references that are in the map. First posted on arXiv 19 Sep 2019.

Builds on

24 of 33 references.

References · 24

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  6. 2017Squeeze-and-Excitation Networkscited by 4,789
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  10. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  11. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  12. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
12 more
  1. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  3. 2017A simple neural network module for relational reasoningcited by 346
  4. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  5. 2015Conditional Computation in Neural Networks for faster modelscited by 167
  6. 2017Adaptive Neural Networks for Efficient Inferencecited by 163
  7. 2017CondenseNet: An Efficient DenseNet using Learned Group Convolutionscited by 122
  8. 2017Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challengecited by 76
  9. 2013Low-Rank Approximations for Conditional Feedforward Computation in Deep Neural Networkscited by 44
  10. 2015Image Question Answering using Convolutional Neural Network with Dynamic Parameter Predictioncited by 26
  11. 2015ABC-CNN: An Attention Based Convolutional Neural Network for Visual Question Answeringcited by 24
  12. 2018Modularity Matters: Learning Invariant Relational Reasoning Taskscited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1.

2 papers.

  1. 2021AdaTag: Multi-Attribute Value Extraction from Product Profiles with Adaptive Decodingcited by 32
  2. 2022Sparsely-gated Mixture-of-Expert Layers for CNN Interpretabilitycited by 14

Similar, not cited

Close in wording, no citation either way

  1. 2025The Quest for Visual Understanding: A Journey Through the Evolution of Visual Question Answering
  2. 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison
  3. 2025Visual question answering: from early developments to recent advances -- a survey
  4. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities
  5. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithms
  6. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge
  7. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
  8. 2023Generative Visual Question Answering