mlsanityData to 7 Oct 2026

Semantic Compositional Networks for Visual Captioning

arXiv 1611.08002 ↗First posted 23 Nov 2016cs.CV cs.CL cs.LGcited by 50

Not on the map. Why?

Abstract (arXiv)

A Semantic Compositional Network (SCN) is developed for image captioning, in which semantic concepts (i.e., tags) are detected from the image, and the probability of each tag is used to compose the parameters in a long short-term memory (LSTM) network. The SCN extends each weight matrix of the LSTM to an ensemble of tag-dependent weight matrices. The degree to which each member of the ensemble is used to generate an image caption is tied to the image-dependent probability of the corresponding tag. In addition to captioning images, we also extend the SCN to generate captions for video clips. We qualitatively analyze semantic composition in SCNs, and quantitatively evaluate the algorithm on three benchmark datasets: COCO, Flickr30k, and Youtube2Text. Experimental results show that the proposed method significantly outperforms prior state-of-the-art approaches, across multiple evaluation metrics.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 19; 2022: 10; 2023: 13; 2024: 2; 2025: 5.

50 papers.

Sort:

By area:

  1. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  2. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
  3. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  4. 2021Chest ImaGenome Dataset for Clinical Reasoningcited by 89
  5. 2022NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Modelscited by 47
  6. 2022Deep Learning Approaches on Image Captioning: A Reviewcited by 43
  7. 2021Injecting Semantic Concepts into End-to-End Image Captioningcited by 41
  8. 2023ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishingcited by 33
  9. 2021Civil Rephrases Of Toxic Texts With Self-Supervised Transformerscited by 32
  10. 2021Towards Accurate Text-based Image Captioning with Content Diversity Explorationcited by 25
  11. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunitiescited by 24
  12. 2023IC3: Image Captioning by Committee Consensuscited by 20
38 more
  1. 2021Adaptive Hierarchical Graph Reasoning with Semantic Coherence for Video-and-Language Inferencecited by 18
  2. 2022Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Cloudscited by 17
  3. 2023Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarkscited by 17
  4. 2022Exploring Discrete Diffusion Models for Image Captioningcited by 16
  5. 2023Transforming Visual Scene Graphs to Image Captionscited by 15
  6. 2021A Survey of Natural Language Generationcited by 13
  7. 2023Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Modelscited by 13
  8. 2021R3Net:Relation-embedded Representation Reconstruction Network for Change Captioningcited by 12
  9. 2023Improving Image Captioning via Predicting Structured Conceptscited by 12
  10. 2022Prophet Attention: Predicting Attention with Future Attention for Image Captioningcited by 10
  11. 2021Controlled Caption Generation for Images Through Adversarial Attackscited by 9
  12. 2021The MSR-Video to Text Dataset with Clean Annotationscited by 8
  13. 2021TCIC: Theme Concepts Learning Cross Language and Vision for Image Captioningcited by 7
  14. 2021End-to-End Video Question-Answer Generation with Generator-Pretester Networkcited by 6
  15. 2021Exploring Semantic Relationships for Unpaired Image Captioningcited by 6
  16. 2021A Comprehensive Review of the Video-to-Text Problemcited by 5
  17. 2021Dense Video Captioning Using Unsupervised Semantic Informationcited by 5
  18. 2025DescribeEarth: Describe Anything for Remote Sensing Imagescited by 5
  19. 2021Cross-Modal Graph with Meta Concepts for Video Captioningcited by 4
  20. 2023Set Prediction Guided by Semantic Concepts for Diverse Video Captioningcited by 4
  21. 2020Nested-Wasserstein Self-Imitation Learning for Sequence Generationcited by 3
  22. 2022Video Captioning: a comparative review of where we are and which could be the routecited by 3
  23. 2023MSVD-Indonesian: A Benchmark for Multimodal Video-Text Tasks in Indonesiancited by 3
  24. 2021Synchronized Audio-Visual Frames with Fractional Positional Encoding for Transformers in Video-to-Text Translationcited by 2
  25. 2022PIC 4th Challenge: Semantic-Assisted Multi-Feature Encoding and Multi-Head Decoding for Dense Video Captioningcited by 2
  26. 2022Diverse Video Captioning by Adaptive Spatio-temporal Attentioncited by 2
  27. 2022Learning to Collocate Visual-Linguistic Neural Modules for Image Captioningcited by 2
  28. 2023A Challenging Multimodal Video Summary: Simultaneously Extracting and Generating Keyframe-Caption Pairs from Videocited by 1
  29. 2025Efficiency Robustness of Dynamic Deep Learning Systemscited by 1
  30. 2025Dense Video Captioning using Graph-based Sentence Summarizationcited by 1
  31. 2021RefineCap: Concept-Aware Refinement for Image Captioningcited by 0
  32. 2022OSIC: A New One-Stage Image Captioner Coinedcited by 0
  33. 2023An Image captioning algorithm based on the Hybrid Deep Learning Technique (CNN+GRU)cited by 0
  34. 2023Stacked Cross-modal Feature Consolidation Attention Networks for Image Captioningcited by 0
  35. 2024Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysiscited by 0
  36. 2024Multi-Modal interpretable automatic video captioningcited by 0
  37. 2025Tri-FusionNet: Enhancing Image Description Generation with Transformer-based Fusion Network and Dual Attention Mechanismcited by 0
  38. 2025Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarizationcited by 0