mlsanityData to 7 Oct 2026

Improved Fusion of Visual and Language Representations by Dense Symmetric Co-Attention for Visual Question Answering

arXiv 1804.00775 ↗First posted 3 Apr 2018cs.CVcited by 51

Not on the map. Why?

Abstract (arXiv)

A key solution to visual question answering (VQA) exists in how to fuse visual and language features extracted from an input image and question. We show that an attention mechanism that enables dense, bi-directional interactions between the two modalities contributes to boost accuracy of prediction of answers. Specifically, we present a simple architecture that is fully symmetric between visual and language representations, in which each question word attends on image regions and each image region attends on question words. It can be stacked to form a hierarchy for multi-step interactions between an image-question pair. We show through experiments that the proposed architecture achieves a new state-of-the-art on VQA and VQA 2.0 despite its small size. We also present qualitative evaluation, demonstrating how the proposed attention mechanism can generate reasonable attention maps on images and questions, which leads to the correct answer prediction.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 15; 2022: 17; 2023: 14; 2024: 4; 2025: 1.

51 papers.

Sort:

By area:

  1. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
  2. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  3. 2023Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Predictioncited by 92
  4. 2022Changer: Feature Interaction is What You Need for Change Detectioncited by 78
  5. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  6. 2022Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Groundingcited by 52
  7. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  8. 2021A Simple Long-Tailed Recognition Baseline via Vision-Language Modelcited by 49
  9. 2022Visual Attention Methods in Deep Learning: An In-Depth Surveycited by 37
  10. 2022MuKEA: Multimodal Knowledge Extraction and Accumulation for Knowledge-based Visual Question Answeringcited by 36
  11. 2021DualVGR: A Dual-Visual Graph Reasoning Unit for Video Question Answeringcited by 35
  12. 2023NuScenes-MQA: Integrated Evaluation of Captions and QA for Autonomous Driving Datasets using Markup Annotationscited by 31
38 more
  1. 2023Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applicationscited by 27
  2. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 26
  3. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  4. 2023When Search Meets Recommendation: Learning Disentangled Search Representation for Recommendationcited by 17
  5. 2021Auto-Parsing Network for Image Captioning and Visual Question Answeringcited by 16
  6. 2021VQA-MHUG: A Gaze Dataset to Study Multimodal Neural Attention in Visual Question Answeringcited by 16
  7. 2022CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-based Autonomous Urban Drivingcited by 16
  8. 2022VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challengescited by 14
  9. 2022MMKGR: Multi-hop Multi-modal Knowledge Graph Reasoningcited by 13
  10. 2021Co-Grounding Networks with Semantic Attention for Referring Expression Comprehension in Videoscited by 11
  11. 2021Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Networkcited by 11
  12. 2022Structured Two-stream Attention Network for Video Question Answeringcited by 9
  13. 2023Co-attention Propagation Network for Zero-Shot Video Object Segmentationcited by 9
  14. 2022Co-VQA : Answering by Interactive Sub Question Sequencecited by 8
  15. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithmscited by 8
  16. 2021Text is no more Enough! A Benchmark for Profile-based Spoken Language Understandingcited by 7
  17. 2021Learning to Recognize Actions on Objects in Egocentric Video with Attention Dictionariescited by 6
  18. 2022Semantic-aware Modular Capsule Routing for Visual Question Answeringcited by 5
  19. 2022A Dual-Attention Learning Network with Word and Sentence Embedding for Medical Visual Question Answeringcited by 5
  20. 2022AlignVE: Visual Entailment Recognition Based on Alignment Relationscited by 5
  21. 2021Bilateral Cross-Modality Graph Matching Attention for Feature Fusion in Visual Question Answeringcited by 4
  22. 2023EAML: Ensemble Self-Attention-based Mutual Learning Network for Document Image Classificationcited by 4
  23. 2021Show, Write, and Retrieve: Entity-aware Article Generation and Retrievalcited by 3
  24. 2024MAST: Video Polyp Segmentation with a Mixture-Attention Siamese Transformercited by 3
  25. 2024SRC-Net: Bi-Temporal Spatial Relationship Concerned Network for Change Detectioncited by 3
  26. 2021Visual Question Answering based on Local-Scene-Aware Referring Expression Generationcited by 2
  27. 2022SceneGATE: Scene-Graph based co-Attention networks for TExt visual question answeringcited by 2
  28. 2023Hierarchical Graph Pattern Understanding for Zero-Shot VOScited by 2
  29. 2021A Better Loss for Visual-Textual Groundingcited by 1
  30. 2022Bilaterally Slimmable Transformer for Elastic and Efficient Visual Question Answeringcited by 1
  31. 2022Do You Know My Emotion? Emotion-Aware Strategy Recognition towards a Persuasive Dialogue Systemcited by 1
  32. 2023Syntax Tree Constrained Graph Network for Visual Question Answeringcited by 1
  33. 2023Spatio-temporal Co-attention Fusion Network for Video Splicing Localizationcited by 1
  34. 2021M2FN: Multi-step Modality Fusion for Advertisement Image Assessmentcited by 0
  35. 2021MMIU: Dataset for Visual Intent Understanding in Multimodal Assistantscited by 0
  36. 2022Localizing Anatomical Landmarks in Ocular Images using Zoom-In Attentive Networkscited by 0
  37. 2023Adaptively Clustering Neighbor Elements for Image-Text Generationcited by 0
  38. 2023Bridge Damage Cause Estimation Using Multiple Images Based on Visual Question Answeringcited by 0

50 most cited of 51 All 51 (JSON)