# Separable Convolutional LSTMs for Faster Video Segmentation

arXiv: https://arxiv.org/abs/1907.06876 · HTML: https://mlsanity.com/paper/1907.06876/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2019.

## Built on by (most cited first, 8 of 8)

- [Recurrent Vision Transformers for Object Detection with Event Cameras](https://mlsanity.com/paper/2212.05598/) (2022)
- [Sequential vessel segmentation via deep channel attention network](https://mlsanity.com/paper/2102.05229/) (2021)
- [Efficient Two-Stream Network for Violence Detection Using Separable Convolutional LSTM](https://mlsanity.com/paper/2102.10590/) (2021)
- [MS-RNN: A Flexible Multi-Scale Framework for Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2206.03010/) (2022)
- [Dynamical system prediction from sparse observations using deep neural networks with Voronoi tessellation and physics constraint](https://mlsanity.com/paper/2409.00458/) (2024)
- [MS-LSTM: Exploring Spatiotemporal Multiscale Representations in Video Prediction Domain](https://mlsanity.com/paper/2304.07724/) (2023)
- [MCDS-VSS: Moving Camera Dynamic Scene Video Semantic Segmentation by Filtering with Self-Supervised Geometry and Motion](https://mlsanity.com/paper/2405.19921/) (2024)
- [BlazeBVD: Make Scale-Time Equalization Great Again for Blind Video Deflickering](https://mlsanity.com/paper/2403.06243/) (2024)

## Abstract (arXiv)

Semantic Segmentation is an important module for autonomous robots such as self-driving cars. The advantage of video segmentation approaches compared to single image segmentation is that temporal image information is considered, and their performance increases due to this. Hence, single image segmentation approaches are extended by recurrent units such as convolutional LSTM (convLSTM) cells, which are placed at suitable positions in the basic network architecture. However, a major critique of video segmentation approaches based on recurrent neural networks is their large parameter count and their computational complexity, and so, their inference time of one video frame takes up to 66 percent longer than their basic version. Inspired by the success of the spatial and depthwise separable convolutional neural networks, we generalize these techniques for convLSTMs in this work, so that the number of parameters and the required FLOPs are reduced significantly. Experiments on different datasets show that the segmentation approaches using the proposed, modified convLSTM cells achieve similar or slightly worse accuracy, but are up to 15 percent faster on a GPU than the ones using the standard convLSTM cells. Furthermore, a new evaluation metric is introduced, which measures the amount of flickering pixels in the segmented video sequence.