# Encoder-Decoder Based Long Short-Term Memory (LSTM) Model for Video Captioning

arXiv: https://arxiv.org/abs/2401.02052 · HTML: https://mlsanity.com/paper/2401.02052/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 21 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 16 of 16)

- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Dense Regression Network for Video Grounding](https://mlsanity.com/paper/2004.03545/) (2020)
- [Sequence to Sequence -- Video to Text](https://mlsanity.com/paper/1505.00487/) (2015)
- [mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video](https://mlsanity.com/paper/2302.00402/) (2023)
- [Labelling unlabelled videos from scratch with multi-modal self-supervision](https://mlsanity.com/paper/2006.13662/) (2020)
- [A Comprehensive Study of Deep Video Action Recognition](https://mlsanity.com/paper/2012.06567/) (2020)
- [Temporally Grounding Language Queries in Videos by Contextual Boundary-aware Prediction](https://mlsanity.com/paper/1909.05010/) (2019)
- [Video Corpus Moment Retrieval with Contrastive Learning](https://mlsanity.com/paper/2105.06247/) (2021)
- [LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval](https://mlsanity.com/paper/1909.13784/) (2019)
- [Visual Grounding in Video for Unsupervised Word Translation](https://mlsanity.com/paper/2003.05078/) (2020)
- [AnaXNet: Anatomy Aware Multi-label Finding Classification in Chest X-ray](https://mlsanity.com/paper/2105.09937/) (2021)
- [Learning Unsupervised Visual Grounding Through Semantic Self-Supervision](https://mlsanity.com/paper/1803.06506/) (2018)
- [Adversarial Sampling for Fairness Testing in Deep Neural Network](https://mlsanity.com/paper/2303.02874/) (2023)
- [Decoupled Spatial Temporal Graphs for Generic Visual Grounding](https://mlsanity.com/paper/2103.10191/) (2021)
- [Enhancing Border Security and Countering Terrorism Through Computer Vision: a Field of Artificial Intelligence](https://mlsanity.com/paper/2303.02869/) (2023)

## Built on by (most cited first, 3 of 3)

- [Deep Learning-Based Speech and Vision Synthesis to Improve Phishing Attack Detection through a Multi-layer Adaptive Framework](https://mlsanity.com/paper/2402.17249/) (2024)
- [An investigation into the performances of the Current state-of-the-art Naive Bayes, Non-Bayesian and Deep Learning Based Classifier for Phishing Detection: A Survey](https://mlsanity.com/paper/2411.16751/) (2024)
- [MAD-OOD: A Deep Learning Cluster-Driven Framework for an Out-of-Distribution Malware Detection and Classification](https://mlsanity.com/paper/2512.17594/) (2025)

## Abstract (arXiv)

This work demonstrates the implementation and use of an encoder-decoder model to perform a many-to-many mapping of video data to text captions. The many-to-many mapping occurs via an input temporal sequence of video frames to an output sequence of words to form a caption sentence. Data preprocessing, model construction, and model training are discussed. Caption correctness is evaluated using 2-gram BLEU scores across the different splits of the dataset. Specific examples of output captions were shown to demonstrate model generality over the video temporal dimension. Predicted captions were shown to generalize over video action, even in instances where the video scene changed dramatically. Model architecture changes are discussed to improve sentence grammar and correctness.