mlsanityData to 7 Oct 2026

Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods

arXiv 1907.09358 ↗First posted 22 Jul 2019cs.CV cs.CL cs.LGcited by 55

Not on the map. Why?

Abstract (arXiv)

Interest in Artificial Intelligence (AI) and its applications has seen unprecedented growth in the last few years. This success can be partly attributed to the advancements made in the sub-fields of AI such as machine learning, computer vision, and natural language processing. Much of the growth in these fields has been made possible with deep learning, a sub-area of machine learning that uses artificial neural networks. This has created significant interest in the integration of vision and language. In this survey, we focus on ten prominent tasks that integrate language and vision by discussing their problem formulation, methods, existing datasets, evaluation measures, and compare the results obtained with corresponding state-of-the-art methods. Our efforts go beyond earlier surveys which are either task-specific or concentrate only on one type of visual content, i.e., image or video. Furthermore, we also provide some potential future directions in this field of research with an anticipation that this survey stimulates innovative thoughts and ideas to address the existing challenges and build new applications.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 19; 2022: 12; 2023: 14; 2024: 8; 2025: 2.

55 papers.

Sort:

By area:

  1. 2022A Survey of Vision-Language Pre-Trained Modelscited by 177
  2. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  3. 2023A Comprehensive Survey on Segment Anything Model for Vision and Beyondcited by 64
  4. 2021Adversarial Text-to-Image Synthesis: A Reviewcited by 52
  5. 2022Vision-and-Language Pretrained Models: A Surveycited by 50
  6. 2021A Review on Explainability in Multimodal Deep Neural Netscited by 46
  7. 2021Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translationcited by 41
  8. 2021Assessing Multilingual Fairness in Pre-trained Multimodal Representationscited by 35
  9. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  10. 2021MMChat: Multi-Modal Chat Dataset on Social Mediacited by 30
  11. 2021Change Detection Meets Visual Question Answeringcited by 29
  12. 2022A Review on Methods and Applications in Multimodal Deep Learningcited by 29
38 more
  1. 2022Vision-Language Intelligence: Tasks, Representation Learning, and Large Modelscited by 28
  2. 2023Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applicationscited by 27
  3. 2024Heterogeneous Contrastive Learning for Foundation Models and Beyondcited by 27
  4. 2021Recent Advances and Trends in Multimodal Deep Learning: A Reviewcited by 26
  5. 2023ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Modelscited by 26
  6. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  7. 2024Would Deep Generative Models Amplify Bias in Future Models?cited by 24
  8. 2023Robust Visual Question Answering: Datasets, Methods, and Future Challengescited by 23
  9. 2021Levels of explainable artificial intelligence for human-aligned conversational explanationscited by 21
  10. 20222D Human Pose Estimation: A Surveycited by 21
  11. 2023Integration and Performance Analysis of Artificial Intelligence and Computer Vision Based on Deep Learning Algorithmscited by 19
  12. 2021Word2Pix: Word to Pixel Cross Attention Transformer in Visual Groundingcited by 16
  13. 2022Toward Explainable and Fine-Grained 3D Grounding through Referring Textual Phrasescited by 16
  14. 2021A Survey of Natural Language Generationcited by 13
  15. 2022A survey on knowledge-enhanced multimodal learningcited by 11
  16. 2023A Video Is Worth 4096 Tokens: Verbalize Videos To Understand Them In Zero Shotcited by 11
  17. 2021CrossMap Transformer: A Crossmodal Masked Path Transformer Using Double Back-Translation for Vision-and-Language Navigationcited by 8
  18. 2025SoccerChat: Integrating Multimodal Data for Enhanced Soccer Game Understandingcited by 8
  19. 2021Cross-Modal Generative Augmentation for Visual Question Answeringcited by 7
  20. 2021Target-dependent UNITER: A Transformer-Based Multimodal Language Comprehension Model for Domestic Service Robotscited by 7
  21. 2023A semantics-driven methodology for high-quality image annotationcited by 6
  22. 2024A Survey on Multi-modal Machine Translation: Tasks, Methods and Challengescited by 6
  23. 2023Switching Head-Tail Funnel UNITER for Dual Referring Expression Comprehension with Fetch-and-Carry Taskscited by 5
  24. 2021Case Relation Transformer: A Crossmodal Language Generation Model for Fetching Instructionscited by 4
  25. 2022Learning More May Not Be Better: Knowledge Transferability in Vision and Language Taskscited by 3
  26. 2022Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Reviewcited by 2
  27. 2022Progressive Fusion for Multimodal Integrationcited by 2
  28. 2022Domain-aware Self-supervised Pre-training for Label-Efficient Meme Analysiscited by 2
  29. 2023The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challengescited by 2
  30. 2023Rethinking the Evaluating Framework for Natural Language Understanding in AI Systems: Language Acquisition as a Core for Future Metricscited by 2
  31. 2021Inferring spatial relations from textual descriptions of imagescited by 1
  32. 2022Fusing Sentence Embeddings Into LSTM-based Autoregressive Language Modelscited by 1
  33. 2023Prototypical Contrastive Transfer Learning for Multimodal Language Understandingcited by 1
  34. 2024Enhancing Scientific Figure Captioning Through Cross-modal Learningcited by 1
  35. 2024VISTA: A Visual and Textual Attention Dataset for Interpreting Multimodal Modelscited by 1
  36. 2024Synergistic Dual Spatial-aware Generation of Image-to-Text and Text-to-Imagecited by 1
  37. 2021Q4EDA: A Novel Strategy for Textual Information Retrieval Based on User Interactions with Visual Representations of Time Seriescited by 0
  38. 2021Classifying Textual Data with Pre-trained Vision Models through Transfer Learning and Data Transformationscited by 0

50 most cited of 55 All 55 (JSON)