# Bidirectional Long-Short Term Memory for Video Description

arXiv: https://arxiv.org/abs/1606.04631 · HTML: https://mlsanity.com/paper/1606.04631/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2016.

## Built on by (most cited first, 4 of 4)

- [A data filling methodology for time series based on CNN and (Bi)LSTM neural networks](https://mlsanity.com/paper/2204.09994/) (2022)
- [Nepali Video Captioning using CNN-RNN Architecture](https://mlsanity.com/paper/2311.02699/) (2023)
- [SGCap: Decoding Semantic Group for Zero-shot Video Captioning](https://mlsanity.com/paper/2508.01270/) (2025)
- [Watching Synthetic Videos: Aligning Cross-modal Representations with Visual Synthesis for Zero-shot Video Captioning](https://mlsanity.com/paper/2608.11013/) (2026)

## Abstract (arXiv)

Video captioning has been attracting broad research attention in multimedia community. However, most existing approaches either ignore temporal information among video frames or just employ local contextual temporal knowledge. In this work, we propose a novel video captioning framework, termed as Bidirectional Long-Short Term Memory (BiLSTM), which deeply captures bidirectional global temporal structure in video. Specifically, we first devise a joint visual modelling approach to encode video data by combining a forward LSTM pass, a backward LSTM pass, together with visual features from Convolutional Neural Networks (CNNs). Then, we inject the derived video representation into the subsequent language model for initialization. The benefits are in two folds: 1) comprehensively preserving sequential and visual information; and 2) adaptively learning dense visual features and sparse semantic representations for videos and sentences, respectively. We verify the effectiveness of our proposed video captioning framework on a commonly-used benchmark, i.e., Microsoft Video Description (MSVD) corpus, and the experimental results demonstrate that the superiority of the proposed approach as compared to several state-of-the-art methods.