# Guiding Long-Short Term Memory for Image Caption Generation

arXiv: https://arxiv.org/abs/1509.04942 · HTML: https://mlsanity.com/paper/1509.04942/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2015.

## Built on by (most cited first, 8 of 8)

- [UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers](https://mlsanity.com/paper/2301.13741/) (2023)
- [CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers](https://mlsanity.com/paper/2305.17455/) (2023)
- [Activating Distributed Visual Region within LLMs for Efficient and Effective Vision-Language Training and Inference](https://mlsanity.com/paper/2412.12785/) (2024)
- [A Secure and Efficient Multi-Object Grasping Detection Approach for Robotic Arms](https://mlsanity.com/paper/2209.03511/) (2022)
- [Multi-modal reward for visual relationships-based image captioning](https://mlsanity.com/paper/2303.10766/) (2023)
- [AIC-AB NET: A Neural Network for Image Captioning with Spatial Attention and Text Attributes](https://mlsanity.com/paper/2307.07370/) (2023)
- [Satellite Captioning: Large Language Models to Augment Labeling](https://mlsanity.com/paper/2312.10905/) (2023)
- [Using Deep Learning to Generate Semantically Correct Hindi Captions](https://mlsanity.com/paper/2602.13352/) (2026)

## Abstract (arXiv)

In this work we focus on the problem of image caption generation. We propose an extension of the long short term memory (LSTM) model, which we coin gLSTM for short. In particular, we add semantic information extracted from the image as extra input to each unit of the LSTM block, with the aim of guiding the model towards solutions that are more tightly coupled to the image content. Additionally, we explore different length normalization strategies for beam search in order to prevent from favoring short sentences. On various benchmark datasets such as Flickr8K, Flickr30K and MS COCO, we obtain results that are on par with or even outperform the current state-of-the-art.