# CNN+CNN: Convolutional Decoders for Image Captioning

arXiv: https://arxiv.org/abs/1805.09019 · HTML: https://mlsanity.com/paper/1805.09019/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2018.

## Built on by (most cited first, 9 of 9)

- [VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning](https://mlsanity.com/paper/2102.10407/) (2021)
- [Deep Learning Approaches on Image Captioning: A Review](https://mlsanity.com/paper/2201.12944/) (2022)
- [Group-based Distinctive Image Captioning with Memory Attention](https://mlsanity.com/paper/2108.09151/) (2021)
- [On Distinctive Image Captioning via Comparing and Reweighting](https://mlsanity.com/paper/2204.03938/) (2022)
- [Improved Bengali Image Captioning via deep convolutional neural network based encoder-decoder model](https://mlsanity.com/paper/2102.07192/) (2021)
- [Knowledge driven Description Synthesis for Floor Plan Interpretation](https://mlsanity.com/paper/2103.08298/) (2021)
- [Multi-Glimpse Network: A Robust and Efficient Classification Architecture based on Recurrent Downsampled Attention](https://mlsanity.com/paper/2111.02018/) (2021)
- [Explainable Image Captioning using CNN- CNN architecture and Hierarchical Attention](https://mlsanity.com/paper/2407.09556/) (2024)
- [Group-based Distinctive Image Captioning with Memory Difference Encoding and Attention](https://mlsanity.com/paper/2504.02496/) (2025)

## Abstract (arXiv)

Image captioning is a challenging task that combines the field of computer vision and natural language processing. A variety of approaches have been proposed to achieve the goal of automatically describing an image, and recurrent neural network (RNN) or long-short term memory (LSTM) based models dominate this field. However, RNNs or LSTMs cannot be calculated in parallel and ignore the underlying hierarchical structure of a sentence. In this paper, we propose a framework that only employs convolutional neural networks (CNNs) to generate captions. Owing to parallel computing, our basic model is around 3 times faster than NIC (an LSTM-based model) during training time, while also providing better results. We conduct extensive experiments on MSCOCO and investigate the influence of the model width and depth. Compared with LSTM-based models that apply similar attention mechanisms, our proposed models achieves comparable scores of BLEU-1,2,3,4 and METEOR, and higher scores of CIDEr. We also test our model on the paragraph annotation dataset, and get higher CIDEr score compared with hierarchical LSTMs