mlsanityData to 7 Oct 2026

Deep Modular Co-Attention Networks for Visual Question Answering

arXiv 1906.10770 ↗First posted 25 Jun 2019cs.CVcited by 338

Not on the map. Why?

Cited by 44 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Visual Question Answering (VQA) requires a fine-grained and simultaneous understanding of both the visual content of images and the textual content of questions. Therefore, designing an effective `co-attention' model to associate key words in questions with key objects in images is central to VQA performance. So far, most successful attempts at co-attention learning have been achieved by using shallow models, and deep co-attention models show little improvement over their shallow counterparts. In this paper, we propose a deep Modular Co-Attention Network (MCAN) that consists of Modular Co-Attention (MCA) layers cascaded in depth. Each MCA layer models the self-attention of questions and images, as well as the guided-attention of images jointly using a modular composition of two basic attention units. We quantitatively and qualitatively evaluate MCAN on the benchmark VQA-v2 dataset and conduct extensive ablation studies to explore the reasons behind MCAN's effectiveness. Experimental results demonstrate that MCAN significantly outperforms the previous state-of-the-art. Our best single model delivers 70.63% overall accuracy on the test-dev set. Code is available at https://github.com/MILVLG/mcan-vqa.

Key facts. Cited by 338 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 72; 2022: 66; 2023: 79; 2024: 63; 2025: 44; 2026: 14.

338 papers.

Sort:

By area:

  1. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  2. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  3. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  4. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  5. 2023A Survey on Multimodal Large Language Modelscited by 898
  6. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  7. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  8. 2023A Comprehensive Overview of Large Language Modelscited by 639
  9. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  10. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 340
  11. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  12. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
38 more
  1. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
  2. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 262
  3. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
  4. 2021GeoQA: A Geometric Question Answering Benchmark Towards Multimodal Numerical Reasoningcited by 224
  5. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
  6. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  7. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 188
  8. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awarenesscited by 178
  9. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  10. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 170
  11. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  12. 2022UniGeo: Unifying Geometry Logical Reasoning via Reformulating Mathematical Expressioncited by 137
  13. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  14. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  15. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  16. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  17. 2022CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailmentcited by 113
  18. 2021Causal Attention for Vision-Language Taskscited by 111
  19. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  20. 2021Interventional Video Grounding with Dual Contrastive Learningcited by 94
  21. 2021Medical Visual Question Answering: A Surveycited by 93
  22. 2022Retrieval Augmented Visual Question Answering with Outside Knowledgecited by 93
  23. 2022Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoningcited by 90
  24. 2021TransRefer3D: Entity-and-Relation Aware Transformer for Fine-Grained 3D Visual Groundingcited by 86
  25. 2022Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectlycited by 86
  26. 2021Multi-Modal Answer Validation for Knowledge-Based VQAcited by 78
  27. 2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 74
  28. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  29. 2022XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understandingcited by 68
  30. 2023Beyond One-to-One: Rethinking the Referring Image Segmentationcited by 68
  31. 2022Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generationcited by 67
  32. 2021UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-trainingcited by 64
  33. 2023Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoningcited by 63
  34. 2022NLX-GPT: A Model for Natural Language Explanations in Vision and Vision-Language Taskscited by 60
  35. 2023T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answeringcited by 59
  36. 2023Read-only Prompt Optimization for Vision-Language Few-shot Learningcited by 58
  37. 2023Large Language Models are Visual Reasoning Coordinatorscited by 58
  38. 2021Human-Adversarial Visual Question Answeringcited by 57

50 most cited of 338 All 338 (JSON)