mlsanityData to 7 Oct 2026

Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding

arXiv 1606.01847 ↗First posted 6 Jun 2016cs.CV cs.AI cs.CLcited by 243

Not on the map. Why?

Abstract (arXiv)

Modeling textual or visual information with vector representations trained from large language or visual datasets has been successfully explored in recent years. However, tasks such as visual question answering require combining these vector representations with each other. Approaches to multimodal pooling include element-wise product or sum, as well as concatenation of the visual and textual representations. We hypothesize that these methods are not as expressive as an outer product of the visual and textual vectors. As the outer product is typically infeasible due to its high dimensionality, we instead propose utilizing Multimodal Compact Bilinear pooling (MCB) to efficiently and expressively combine multimodal features. We extensively evaluate MCB on the visual question answering and grounding tasks. We consistently show the benefit of MCB over ablations without MCB. For visual question answering, we present an architecture which uses MCB twice, once for predicting attention over spatial features and again to combine the attended representation with the question representation. This model outperforms the state-of-the-art on the Visual7W dataset and the VQA challenge.

Key facts. Cited by 243 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 69; 2022: 64; 2023: 42; 2024: 30; 2025: 30; 2026: 8.

243 papers.

Sort:

By area:

  1. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  2. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  3. 2022Versatile Diffusion: Text, Images and Variations All in One Diffusion Modelcited by 199
  4. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 157
  5. 2021Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphscited by 144
  6. 2021Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Cloudscited by 128
  7. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 121
  8. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoningcited by 119
  9. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  10. 2024VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agentscited by 101
  11. 2022Exploiting Unlabeled Data with Vision and Language Models for Object Detectioncited by 97
  12. 2021Medical Visual Question Answering: A Surveycited by 93
38 more
  1. 2023Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Predictioncited by 92
  2. 2022Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoningcited by 90
  3. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  4. 2022Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectlycited by 86
  5. 2021e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Taskscited by 84
  6. 2023Quantifying & Modeling Multimodal Interactions: An Information Decomposition Frameworkcited by 81
  7. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  8. 2022Multi-modal Sensor Fusion for Auto Driving Perception: A Surveycited by 73
  9. 2021Goal-Oriented Gaze Estimation for Zero-Shot Learningcited by 68
  10. 2021Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answeringcited by 62
  11. 2022NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Taskscited by 60
  12. 2021Human-Adversarial Visual Question Answeringcited by 57
  13. 2022Hate-CLIPper: Multimodal Hateful Meme Classification based on Cross-modal Interaction of CLIP Featurescited by 55
  14. 2021Multiple Meta-model Quantifying for Medical Visual Question Answeringcited by 54
  15. 2022Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Groundingcited by 52
  16. 2024Toward Visual Grounding: A Surveycited by 52
  17. 2022Divert More Attention to Vision-Language Trackingcited by 47
  18. 2021Biomedical Question Answering: A Survey of Approaches and Challengescited by 46
  19. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  20. 2024Exploring Diverse Methods in Visual Question Answeringcited by 45
  21. 2021Grounding 'Grounding' in NLPcited by 44
  22. 2021MM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognitioncited by 44
  23. 2021Spatial Dual-Modality Graph Reasoning for Key Information Extractioncited by 43
  24. 2022Deep Learning Approaches on Image Captioning: A Reviewcited by 43
  25. 2021Visual Grounding with Transformerscited by 42
  26. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  27. 2023BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Modelscited by 41
  28. 2022SwapMix: Diagnosing and Regularizing the Over-Reliance on Visual Context in Visual Question Answeringcited by 38
  29. 2022From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Datacited by 38
  30. 2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
  31. 2021DualVGR: A Dual-Visual Graph Reasoning Unit for Video Question Answeringcited by 35
  32. 2023Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checkingcited by 34
  33. 2022Learning to Answer Visual Questions from Web Videoscited by 30
  34. 2024HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognitioncited by 30
  35. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 29
  36. 2021Multimodal Entity Linking for Tweetscited by 28
  37. 2021Consensus Graph Representation Learning for Better Grounded Image Captioningcited by 28
  38. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28

50 most cited of 243 All 243 (JSON)