mlsanityData to 7 Oct 2026

Boosting Image Captioning with Attributes

arXiv 1611.01646 ↗First posted 5 Nov 2016cs.CVcited by 76

Not on the map. Why?

Abstract (arXiv)

Automatically describing an image with a natural language has been an emerging challenge in both fields of computer vision and natural language processing. In this paper, we present Long Short-Term Memory with Attributes (LSTM-A) - a novel architecture that integrates attributes into the successful Convolutional Neural Networks (CNNs) plus Recurrent Neural Networks (RNNs) image captioning framework, by training them in an end-to-end manner. To incorporate attributes, we construct variants of architectures by feeding image representations and attributes into RNNs in different ways to explore the mutual but also fuzzy relationship between them. Extensive experiments are conducted on COCO image captioning dataset and our framework achieves superior results when compared to state-of-the-art deep models. Most remarkably, we obtain METEOR/CIDEr-D of 25.2%/98.6% on testing data of widely used and publicly available splits in (Karpathy & Fei-Fei, 2015) when extracting image representations by GoogleNet and achieve to date top-1 performance on COCO captioning Leaderboard.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 27; 2023: 11; 2024: 5; 2025: 6; 2026: 1.

76 papers.

Sort:

By area:

  1. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  2. 2023To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuningcited by 143
  3. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
  4. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  5. 2023Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluationcited by 78
  6. 2022GRIT: Faster and Better Image captioning Transformer Using Dual Visual Featurescited by 56
  7. 2021CPTR: Full Transformer Network for Image Captioningcited by 55
  8. 2023One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Eracited by 55
  9. 2021From General to Specific: Informative Scene Graph Generation via Balance Adjustmentcited by 51
  10. 2022Deep Learning Approaches on Image Captioning: A Reviewcited by 43
  11. 2022Semantic-Conditional Diffusion Networks for Image Captioningcited by 38
  12. 2022Comprehending and Ordering Semantics for Image Captioningcited by 33
38 more
  1. 2022M^4I: Multi-modal Models Membership Inferencecited by 30
  2. 2021Conversational Fashion Image Retrieval via Multiturn Natural Language Feedbackcited by 29
  3. 2021A Survey on Temporal Sentence Grounding in Videoscited by 29
  4. 2021Dual Graph Convolutional Networks with Transformer and Curriculum Learning for Image Captioningcited by 27
  5. 2023Towards Local Visual Modeling for Image Captioningcited by 26
  6. 2022Prompt-based Learning for Unpaired Image Captioningcited by 23
  7. 2022Entity-enhanced Adaptive Reconstruction Network for Weakly Supervised Referring Expression Groundingcited by 22
  8. 2023Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipationcited by 21
  9. 2021Scheduled Sampling in Vision-Language Pretraining with Decoupled Encoder-Decoder Networkcited by 17
  10. 2021Detector-Free Weakly Supervised Grounding by Separationcited by 16
  11. 2021Auto-Parsing Network for Image Captioning and Visual Question Answeringcited by 16
  12. 2022Exploring Discrete Diffusion Models for Image Captioningcited by 16
  13. 2022Cross-modal Semantic Enhanced Interaction for Image-Sentence Retrievalcited by 14
  14. 2021SGNet: A Super-class Guided Network for Image Classification and Object Detectioncited by 13
  15. 2021Neural Attention for Image Captioning: Review of Outstanding Methodscited by 11
  16. 2021X-modaler: A Versatile and High-performance Codebase for Cross-modal Analyticscited by 10
  17. 2022Prophet Attention: Predicting Attention with Future Attention for Image Captioningcited by 10
  18. 2022Boosting Video-Text Retrieval with Explicit High-Level Semanticscited by 8
  19. 2021Similar Scenes arouse Similar Emotions: Parallel Data Augmentation for Stylized Image Captioningcited by 7
  20. 2022Image Captioning via Compact Bidirectional Architecturecited by 7
  21. 2022Uni-EDEN: Universal Encoder-Decoder Network by Multi-Granular Vision-Language Pre-trainingcited by 7
  22. 2022On Distinctive Image Captioning via Comparing and Reweightingcited by 7
  23. 2022Learning To Generate Scene Graph from Head to Tailcited by 7
  24. 2021Exploring Semantic Relationships for Unpaired Image Captioningcited by 6
  25. 2022Distinctive Image Captioning via CLIP Guided Group Optimizationcited by 6
  26. 2024Unleashing Text-to-Image Diffusion Prior for Zero-Shot Image Captioningcited by 6
  27. 2021Saying the Unseen: Video Descriptions via Dialog Agentscited by 5
  28. 2022Efficient Modeling of Future Context for Image Captioningcited by 5
  29. 2025DescribeEarth: Describe Anything for Remote Sensing Imagescited by 5
  30. 2022DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attentioncited by 4
  31. 2024Enhancing Multimodal Large Language Models with Multi-instance Visual Prompt Generator for Visual Representation Enrichmentcited by 4
  32. 2021A Picture May Be Worth a Hundred Words for Visual Question Answeringcited by 3
  33. 2021Label-Attention Transformer with Geometrically Coherent Objects for Image Captioningcited by 3
  34. 2022Image Captioning In the Transformer Agecited by 3
  35. 2024Image Captioning via Dynamic Path Customizationcited by 3
  36. 2021Image Captioning using Multiple Transformers for Self-Attention Mechanismcited by 2
  37. 2021Knowledge driven Description Synthesis for Floor Plan Interpretationcited by 2
  38. 2022Guiding Attention using Partial-Order Relationships for Image Captioningcited by 2

50 most cited of 76 All 76 (JSON)