# Cubic LSTMs for Video Prediction

arXiv: https://arxiv.org/abs/1904.09412 · HTML: https://mlsanity.com/paper/1904.09412/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Apr 2019.

## Built on by (most cited first, 30 of 32)

- [SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustness](https://mlsanity.com/paper/2207.12391/) (2022)
- [Adaptive Context Selection for Polyp Segmentation](https://mlsanity.com/paper/2301.04799/) (2023)
- [nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Model](https://mlsanity.com/paper/2402.03526/) (2024)
- [Enhanced U-Net: A Feature Enhancement Network for Polyp Segmentation](https://mlsanity.com/paper/2105.00999/) (2021)
- [Diffusion-based Data Augmentation for Nuclei Image Segmentation](https://mlsanity.com/paper/2310.14197/) (2023)
- [More than Encoder: Introducing Transformer Decoder to Upsample](https://mlsanity.com/paper/2106.10637/) (2021)
- [Prompt-Matched Semantic Segmentation](https://mlsanity.com/paper/2208.10159/) (2022)
- [Center Smoothing: Certified Robustness for Networks with Structured Outputs](https://mlsanity.com/paper/2102.09701/) (2021)
- [Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challenges](https://mlsanity.com/paper/2303.14133/) (2023)
- [Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging](https://mlsanity.com/paper/2107.09543/) (2021)
- [Adversarial Examples on Segmentation Models Can be Easy to Transfer](https://mlsanity.com/paper/2111.11368/) (2021)
- [Rethinking Query-based Transformer for Continual Image Segmentation](https://mlsanity.com/paper/2507.07831/) (2025)
- [Robust Real-World Image Super-Resolution against Adversarial Attacks](https://mlsanity.com/paper/2208.00428/) (2022)
- [TranSegPGD: Improving Transferability of Adversarial Examples on Semantic Segmentation](https://mlsanity.com/paper/2312.02207/) (2023)
- [Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context](https://mlsanity.com/paper/2510.20229/) (2025)
- [Brain-inspired Artificial Intelligence: A Comprehensive Review](https://mlsanity.com/paper/2408.14811/) (2024)
- [Noise Optimization for Artificial Neural Networks](https://mlsanity.com/paper/2102.04450/) (2021)
- [Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formats](https://mlsanity.com/paper/2511.17254/) (2025)
- [A Certified Radius-Guided Attack Framework to Image Segmentation Models](https://mlsanity.com/paper/2304.02693/) (2023)
- [Uncertainty-based Detection of Adversarial Attacks in Semantic Segmentation](https://mlsanity.com/paper/2305.12825/) (2023)
- [Visual-Attribute Prompt Learning for Progressive Mild Cognitive Impairment Prediction](https://mlsanity.com/paper/2310.14158/) (2023)
- [View-Disentangled Transformer for Brain Lesion Detection](https://mlsanity.com/paper/2209.09657/) (2022)
- [Adversarial Medical Image with Hierarchical Feature Hiding](https://mlsanity.com/paper/2312.01679/) (2023)
- [No More Sibling Rivalry: Debiasing Human-Object Interaction Detection](https://mlsanity.com/paper/2509.00760/) (2025)
- [Co-Seg: An Image Segmentation Framework Against Label Corruption](https://mlsanity.com/paper/2102.00523/) (2021)
- [Towards Better Certified Segmentation via Diffusion Models](https://mlsanity.com/paper/2306.09949/) (2023)
- [Atlas-aware ConvNetfor Accurate yet Robust Anatomical Segmentation](https://mlsanity.com/paper/2102.01256/) (2021)
- [Medical Aegis: Robust adversarial protectors for medical images](https://mlsanity.com/paper/2111.10969/) (2021)
- [Slice Imputation: Intermediate Slice Interpolation for Anisotropic 3D Medical Image Segmentation](https://mlsanity.com/paper/2203.10773/) (2022)
- [Wildfire Forecasting with Satellite Images and Deep Generative Model](https://mlsanity.com/paper/2208.09411/) (2022)

## Abstract (arXiv)

Predicting future frames in videos has become a promising direction of research for both computer vision and robot learning communities. The core of this problem involves moving object capture and future motion prediction. While object capture specifies which objects are moving in videos, motion prediction describes their future dynamics. Motivated by this analysis, we propose a Cubic Long Short-Term Memory (CubicLSTM) unit for video prediction. CubicLSTM consists of three branches, i.e., a spatial branch for capturing moving objects, a temporal branch for processing motions, and an output branch for combining the first two branches to generate predicted frames. Stacking multiple CubicLSTM units along the spatial branch and output branch, and then evolving along the temporal branch can form a cubic recurrent neural network (CubicRNN). Experiment shows that CubicRNN produces more accurate video predictions than prior methods on both synthetic and real-world datasets.