mlsanityData to 14 Oct 2026

Dual Attention Networks for Multimodal Reasoning and Matching

arXiv 1611.00471 ↗First posted 2 Nov 2016cs.CVcited by 114

Not on the map. Why?

Abstract (arXiv)

We propose Dual Attention Networks (DANs) which jointly leverage visual and textual attention mechanisms to capture fine-grained interplay between vision and language. DANs attend to specific regions in images and words in text through multiple steps and gather essential information from both modalities. Based on this framework, we introduce two types of DANs for multimodal reasoning and matching, respectively. The reasoning model allows visual and textual attentions to steer each other during collaborative inference, which is useful for tasks such as Visual Question Answering (VQA). In addition, the matching model exploits the two attention mechanisms to estimate the similarity between images and sentences by focusing on their shared semantics. Our extensive experiments validate the effectiveness of DANs in combining vision and language, achieving the state-of-the-art performance on public benchmarks for VQA and image-text matching.

Key facts. Cited by 114 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 44; 2022: 24; 2023: 20; 2024: 14; 2025: 10; 2026: 2.

114 papers.

Sort:

By area:

  1. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  2. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 288
  3. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  4. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  5. 2021Probabilistic Embeddings for Cross-Modal Retrievalcited by 151
  6. 2021Similarity Reasoning and Filtration for Image-Text Matchingcited by 121
  7. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  8. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  9. 2021DSSL: Deep Surroundings-person Separation Learning for Text-based Person Retrievalcited by 102
  10. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 76
  11. 2021Adaptive Multi-Teacher Multi-level Knowledge Distillationcited by 72
  12. 2021Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answeringcited by 56
38 more
  1. 2024Toward Visual Grounding: A Surveycited by 52
  2. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 51
  3. 2021Bi-Temporal Semantic Reasoning for the Semantic Change Detection in HR Remote Sensing Imagescited by 43
  4. 2023BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Modelscited by 41
  5. 2022CAMS: An Annotated Corpus for Causal Analysis of Mental Health Issues in Social Media Postscited by 34
  6. 2024PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamicscited by 34
  7. 2022Improving Cross-Modal Retrieval with Set of Diverse Embeddingscited by 33
  8. 2021Spatio-Temporal Representation Factorization for Video-based Person Re-Identificationcited by 29
  9. 2021Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrievalcited by 28
  10. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28
  11. 2022Beyond Fixation: Dynamic Window Visual Transformercited by 28
  12. 2021Looking Outside the Window: Wide-Context Transformer for the Semantic Segmentation of High-Resolution Remote Sensing Imagescited by 27
  13. 2023Multi-modal Machine Learning in Engineering Design: A Review and Future Directionscited by 27
  14. 2021Step-Wise Hierarchical Alignment Network for Image-Text Matchingcited by 26
  15. 2023Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directionscited by 25
  16. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 25
  17. 2022Multimodal Pre-training Based on Graph Attention Network for Document Understandingcited by 24
  18. 2021Structured Multi-modal Feature Embedding and Alignment for Image-Sentence Retrievalcited by 23
  19. 2021Multimodal Integration of Human-Like Attention in Visual Question Answeringcited by 20
  20. 2023DA-TransUNet: Integrating Spatial and Channel Dual Attention with Transformer U-Net for Medical Image Segmentationcited by 16
  21. 2021VisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-wordscited by 14
  22. 2022VLCDoC: Vision-Language Contrastive Pre-Training Model for Cross-Modal Document Classificationcited by 14
  23. 2022Cross-modal Semantic Enhanced Interaction for Image-Sentence Retrievalcited by 14
  24. 2023Direction-Oriented Visual-semantic Embedding Model for Remote Sensing Image-text Retrievalcited by 14
  25. 2021Is An Image Worth Five Sentences? A New Look into Semantics for Image-Text Matchingcited by 13
  26. 2021Two-stage Visual Cues Enhancement Network for Referring Image Segmentationcited by 13
  27. 2021Telling the What while Pointing to the Where: Multimodal Queries for Image Retrievalcited by 12
  28. 2021Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learningcited by 12
  29. 2024Multimodal Learned Sparse Retrieval with Probabilistic Expansion Controlcited by 12
  30. 2021ICECAP: Information Concentrated Entity-aware Image Captioningcited by 11
  31. 2023Plug-and-Play Regulators for Image-Text Matchingcited by 10
  32. 2023Rethinking Benchmarks for Cross-modal Image-text Retrievalcited by 10
  33. 2023Multimodal Graph Transformer for Multimodal Question Answeringcited by 10
  34. 2022Structured Two-stream Attention Network for Video Question Answeringcited by 9
  35. 2023BIG-C: a Multimodal Multi-Purpose Dataset for Bembacited by 9
  36. 2024FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wildcited by 9
  37. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  38. 2021Graph-Based Tri-Attention Network for Answer Ranking in CQAcited by 8

50 most cited of 114 All 114 (JSON)