mlsanityData to 7 Oct 2026

Pixels to Prose: Understanding the art of Image Captioning

arXiv 2408.15714 ↗First posted 28 Aug 2024cs.CV cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

In the era of evolving artificial intelligence, machines are increasingly emulating human-like capabilities, including visual perception and linguistic expression. Image captioning stands at the intersection of these domains, enabling machines to interpret visual content and generate descriptive text. This paper provides a thorough review of image captioning techniques, catering to individuals entering the field of machine learning who seek a comprehensive understanding of available options, from foundational methods to state-of-the-art approaches. Beginning with an exploration of primitive architectures, the review traces the evolution of image captioning models to the latest cutting-edge solutions. By dissecting the components of these architectures, readers gain insights into the underlying mechanisms and can select suitable approaches tailored to specific problem requirements without duplicating efforts. The paper also delves into the application of image captioning in the medical domain, illuminating its significance in various real-world scenarios. Furthermore, the review offers guidance on evaluating the performance of image captioning systems, highlighting key metrics for assessment. By synthesizing theoretical concepts with practical application, this paper equips readers with the knowledge needed to navigate the complex landscape of image captioning and harness its potential for diverse applications in machine learning and beyond.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 84 of its 135 references that are in the map. First posted on arXiv 28 Aug 2024.

Builds on

84 of 135 references.

References · 84

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2015Fast R-CNNcited by 2,930
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  10. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  11. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  12. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
38 more
  1. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  2. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
  3. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  4. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  5. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  6. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  7. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  8. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  9. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
  10. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  11. 2014Show and Tell: A Neural Image Caption Generatorcited by 877
  12. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
  13. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  14. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  15. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  16. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  17. 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 483
  18. 2018nocaps: novel object captioning at scalecited by 444
  19. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  20. 2016Tutorial on Variational Autoencoderscited by 404
  21. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 398
  22. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
  23. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  24. 2015Visual7W: Grounded Question Answering in Imagescited by 315
  25. 2019Attention on Attention for Image Captioningcited by 289
  26. 2020Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformerscited by 288
  27. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  28. 2018Auto-Encoding Scene Graphs for Image Captioningcited by 254
  29. 2016Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioningcited by 231
  30. 2018Exploring Visual Relationship for Image Captioningcited by 215
  31. 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 213
  32. 2019A Survey of the Recent Architectures of Deep Convolutional Neural Networkscited by 211
  33. 2020X-Linear Attention Networks for Image Captioningcited by 198
  34. 2014Unifying Visual-Semantic Embeddings with Multimodal Neural Language Modelscited by 197
  35. 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
  36. 2019Image Captioning: Transforming Objects into Wordscited by 171
  37. 2017Scene Graph Generation from Objects, Phrases and Region Captionscited by 154
  38. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.