# Phrase-based Image Captioning

arXiv: https://arxiv.org/abs/1502.03671 · HTML: https://mlsanity.com/paper/1502.03671/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2015.

## Built on by (most cited first, 6 of 6)

- [Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions](https://mlsanity.com/paper/2209.03430/) (2022)
- [Sensor-Augmented Egocentric-Video Captioning with Dynamic Modal Attention](https://mlsanity.com/paper/2109.02955/) (2021)
- [Unpaired Image Captioning by Image-level Weakly-Supervised Visual Concept Recognition](https://mlsanity.com/paper/2203.03195/) (2022)
- [Foundations of Multisensory Artificial Intelligence](https://mlsanity.com/paper/2404.18976/) (2024)
- [Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysis](https://mlsanity.com/paper/2408.04909/) (2024)
- [Compressed Image Captioning using CNN-based Encoder-Decoder Framework](https://mlsanity.com/paper/2404.18062/) (2024)

## Abstract (arXiv)

Generating a novel textual description of an image is an interesting problem that connects computer vision and natural language processing. In this paper, we present a simple model that is able to generate descriptive sentences given a sample image. This model has a strong focus on the syntax of the descriptions. We train a purely bilinear model that learns a metric between an image representation (generated from a previously trained Convolutional Neural Network) and phrases that are used to described them. The system is then able to infer phrases from a given image sample. Based on caption syntax statistics, we propose a simple language model that can produce relevant descriptions for a given test image using the phrases inferred. Our approach, which is considerably simpler than state-of-the-art models, achieves comparable results in two popular datasets for the task: Flickr30k and the recently proposed Microsoft COCO.