mlsanityData to 7 Oct 2026

What value do explicit high level concepts have in vision to language problems?

arXiv 1506.01144 ↗First posted 3 Jun 2015cs.CVcited by 31

Not on the map. Why?

Abstract (arXiv)

Much of the recent progress in Vision-to-Language (V2L) problems has been achieved through a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). This approach does not explicitly represent high-level semantic concepts, but rather seeks to progress directly from image features to text. We propose here a method of incorporating high-level concepts into the very successful CNN-RNN approach, and show that it achieves a significant improvement on the state-of-the-art performance in both image captioning and visual question answering. We also show that the same mechanism can be used to introduce external semantic information and that doing so further improves performance. In doing so we provide an analysis of the value of high level semantic information in V2L problems.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 10; 2022: 11; 2023: 4; 2024: 4; 2025: 2.

31 papers.

Sort:

By area:

  1. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
  2. 2021MOMENTA: A Multimodal Framework for Detecting Harmful Memes and Their Targetscited by 85
  3. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  4. 2021Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoningcited by 68
  5. 2021Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Languagecited by 55
  6. 2022Deep Learning Approaches on Image Captioning: A Reviewcited by 43
  7. 2022Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioningcited by 25
  8. 2021ITA: Image-Text Alignments for Multi-Modal Named Entity Recognitioncited by 24
  9. 2022Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignmentcited by 19
  10. 2021O2NA: An Object-Oriented Non-Autoregressive Approach for Controllable Video Captioningcited by 18
  11. 2022Exploring Discrete Diffusion Models for Image Captioningcited by 16
  12. 2022Aligning Source Visual and Target Language Domains for Unpaired Video Captioningcited by 14
19 more
  1. 2021A Survey of Natural Language Generationcited by 13
  2. 2023Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluationcited by 12
  3. 2022Boosting Video-Text Retrieval with Explicit High-Level Semanticscited by 8
  4. 2021Exploring Semantic Relationships for Unpaired Image Captioningcited by 6
  5. 2022FACT: Learning Governing Abstractions Behind Integer Sequencescited by 6
  6. 2024Beyond Unimodal Learning: The Importance of Integrating Multiple Modalities for Lifelong Learningcited by 5
  7. 2025DescribeEarth: Describe Anything for Remote Sensing Imagescited by 5
  8. 2021Cross-Modal Graph with Meta Concepts for Video Captioningcited by 4
  9. 2022DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attentioncited by 4
  10. 2024Reminding Multimodal Large Language Models of Object-aware Knowledge with Retrieved Tagscited by 2
  11. 2025Coreset Selection via LLM-based Concept Bottleneckscited by 2
  12. 2022A Comprehensive Survey of Natural Language Generation Advances from the Perspective of Digital Deceptioncited by 1
  13. 2022How to Describe Images in a More Funny Way? Towards a Modular Approach to Cross-Modal Sarcasm Generationcited by 1
  14. 2024Pixels to Prose: Understanding the art of Image Captioningcited by 1
  15. 2021RefineCap: Concept-Aware Refinement for Image Captioningcited by 0
  16. 2022An Integrated Approach for Video Captioning and Applicationscited by 0
  17. 2023An Image captioning algorithm based on the Hybrid Deep Learning Technique (CNN+GRU)cited by 0
  18. 2023Stacked Cross-modal Feature Consolidation Attention Networks for Image Captioningcited by 0
  19. 2023Multi-modal reward for visual relationships-based image captioningcited by 0