Pixels to Prose: Understanding the art of Image Captioning
Not on the map. Why?
Abstract (arXiv)
In the era of evolving artificial intelligence, machines are increasingly emulating human-like capabilities, including visual perception and linguistic expression. Image captioning stands at the intersection of these domains, enabling machines to interpret visual content and generate descriptive text. This paper provides a thorough review of image captioning techniques, catering to individuals entering the field of machine learning who seek a comprehensive understanding of available options, from foundational methods to state-of-the-art approaches. Beginning with an exploration of primitive architectures, the review traces the evolution of image captioning models to the latest cutting-edge solutions. By dissecting the components of these architectures, readers gain insights into the underlying mechanisms and can select suitable approaches tailored to specific problem requirements without duplicating efforts. The paper also delves into the application of image captioning in the medical domain, illuminating its significance in various real-world scenarios. Furthermore, the review offers guidance on evaluating the performance of image captioning systems, highlighting key metrics for assessment. By synthesizing theoretical concepts with practical application, this paper equips readers with the knowledge needed to navigate the complex landscape of image captioning and harness its potential for diverse applications in machine learning and beyond.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 84 of its 135 references that are in the map. First posted on arXiv 28 Aug 2024.
Builds on
84 of 135 references.
References · 84
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2016Densely Connected Convolutional Networkscited by 6,691
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2015Fast R-CNNcited by 2,930
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
- 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
38 more
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,027
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
- 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
- 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
- 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
- 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 966
- 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
- 2014Show and Tell: A Neural Image Caption Generatorcited by 877
- 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 820
- 2021Florence: A New Foundation Model for Computer Visioncited by 802
- 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
- 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
- 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
- 2014Long-term Recurrent Convolutional Networks for Visual Recognition and Descriptioncited by 483
- 2018nocaps: novel object captioning at scalecited by 444
- 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
- 2016Tutorial on Variational Autoencoderscited by 404
- 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 398
- 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
- 2019Meshed-Memory Transformer for Image Captioningcited by 343
- 2015Visual7W: Grounded Question Answering in Imagescited by 315
- 2019Attention on Attention for Image Captioningcited by 289
- 2020Pixel-BERT: Aligning Image Pixels with Text by Deep Multi-Modal Transformerscited by 288
- 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
- 2018Auto-Encoding Scene Graphs for Image Captioningcited by 254
- 2016Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioningcited by 231
- 2018Exploring Visual Relationship for Image Captioningcited by 215
- 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 213
- 2019A Survey of the Recent Architectures of Deep Convolutional Neural Networkscited by 211
- 2020X-Linear Attention Networks for Image Captioningcited by 198
- 2014Unifying Visual-Semantic Embeddings with Multimodal Neural Language Modelscited by 197
- 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
- 2019Image Captioning: Transforming Objects into Wordscited by 171
- 2017Scene Graph Generation from Objects, Phrases and Region Captionscited by 154
- 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
50 most cited of 84 All 84 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.