# Convolutional Image Captioning

arXiv: https://arxiv.org/abs/1711.09151 · HTML: https://mlsanity.com/paper/1711.09151/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2017.

## Built on by (most cited first, 30 of 52)

- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic](https://mlsanity.com/paper/2111.14447/) (2021)
- [From Show to Tell: A Survey on Deep Learning-based Image Captioning](https://mlsanity.com/paper/2107.06912/) (2021)
- [Vision-Language Pre-training: Basics, Recent Advances, and Future Trends](https://mlsanity.com/paper/2210.09263/) (2022)
- [Context-Guided Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2401.01578/) (2024)
- [Deep Learning Approaches on Image Captioning: A Review](https://mlsanity.com/paper/2201.12944/) (2022)
- [MMDocIR: Benchmarking Multimodal Retrieval for Long Documents](https://mlsanity.com/paper/2501.08828/) (2025)
- [Retrieval-Augmented Transformer for Image Captioning](https://mlsanity.com/paper/2207.13162/) (2022)
- [Measuring Representational Harms in Image Captioning](https://mlsanity.com/paper/2206.07173/) (2022)
- [M^4I: Multi-modal Models Membership Inference](https://mlsanity.com/paper/2209.06997/) (2022)
- [Multi-modal Machine Learning in Engineering Design: A Review and Future Directions](https://mlsanity.com/paper/2302.10909/) (2023)
- [Survey of Social Bias in Vision-Language Models](https://mlsanity.com/paper/2309.14381/) (2023)
- [ReFormer: The Relational Transformer for Image Captioning](https://mlsanity.com/paper/2107.14178/) (2021)
- [Dynamic Attention guided Multi-Trajectory Analysis for Single Object Tracking](https://mlsanity.com/paper/2103.16086/) (2021)
- [MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection](https://mlsanity.com/paper/2307.02733/) (2023)
- [Journalistic Guidelines Aware News Image Captioning](https://mlsanity.com/paper/2109.02865/) (2021)
- [CaMEL: Mean Teacher Learning for Image Captioning](https://mlsanity.com/paper/2202.10492/) (2022)
- [Group-based Distinctive Image Captioning with Memory Attention](https://mlsanity.com/paper/2108.09151/) (2021)
- [Neural Attention for Image Captioning: Review of Outstanding Methods](https://mlsanity.com/paper/2111.15015/) (2021)
- [Diagnosing Human-object Interaction Detectors](https://mlsanity.com/paper/2308.08529/) (2023)
- [X-modaler: A Versatile and High-performance Codebase for Cross-modal Analytics](https://mlsanity.com/paper/2108.08217/) (2021)
- [Discrete-continuous Action Space Policy Gradient-based Attention for Image-Text Matching](https://mlsanity.com/paper/2104.10406/) (2021)
- [Multi-Modal Image Captioning for the Visually Impaired](https://mlsanity.com/paper/2105.08106/) (2021)
- [Automatic Generation of Descriptive Titles for Video Clips Using Deep Learning](https://mlsanity.com/paper/2104.03337/) (2021)
- [On Distinctive Image Captioning via Comparing and Reweighting](https://mlsanity.com/paper/2204.03938/) (2022)
- [Beyond Greedy Search: Tracking by Multi-Agent Reinforcement Learning-based Beam Search](https://mlsanity.com/paper/2205.09676/) (2022)
- [Image Captioning using Deep Stacked LSTMs, Contextual Word Embeddings and Data Augmentation](https://mlsanity.com/paper/2102.11237/) (2021)
- [Bornon: Bengali Image Captioning with Transformer-based Deep learning approach](https://mlsanity.com/paper/2109.05218/) (2021)
- [Improving Radiology Summarization with Radiograph and Anatomy Prompts](https://mlsanity.com/paper/2210.08303/) (2022)
- [AI Readiness in Healthcare through Storytelling XAI](https://mlsanity.com/paper/2410.18725/) (2024)

## Abstract (arXiv)

Image captioning is an important but challenging task, applicable to virtual assistants, editing tools, image indexing, and support of the disabled. Its challenges are due to the variability and ambiguity of possible image descriptions. In recent years significant progress has been made in image captioning, using Recurrent Neural Networks powered by long-short-term-memory (LSTM) units. Despite mitigating the vanishing gradient problem, and despite their compelling ability to memorize dependencies, LSTM units are complex and inherently sequential across time. To address this issue, recent work has shown benefits of convolutional networks for machine translation and conditional image generation. Inspired by their success, in this paper, we develop a convolutional image captioning technique. We demonstrate its efficacy on the challenging MSCOCO dataset and demonstrate performance on par with the baseline, while having a faster training time per number of parameters. We also perform a detailed analysis, providing compelling reasons in favor of convolutional language generation approaches.