# An Efficient 3D CNN for Action/Object Segmentation in Video

arXiv: https://arxiv.org/abs/1907.08895 · HTML: https://mlsanity.com/paper/1907.08895/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2019.

## Built on by (most cited first, 10 of 10)

- [D^2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos](https://mlsanity.com/paper/2111.07774/) (2021)
- [CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos](https://mlsanity.com/paper/2505.18561/) (2025)
- [AutoVideo: An Automated Video Action Recognition System](https://mlsanity.com/paper/2108.04212/) (2021)
- [Cross-Modal learning for Audio-Visual Video Parsing](https://mlsanity.com/paper/2104.04598/) (2021)
- [Lightweight Multi-Frame Integration for Robust YOLO Object Detection in Videos](https://mlsanity.com/paper/2506.20550/) (2025)
- [Non-linear Motion Estimation for Video Frame Interpolation using Space-time Convolutions](https://mlsanity.com/paper/2201.11407/) (2022)
- [TransNet: A Transfer Learning-Based Network for Human Action Recognition](https://mlsanity.com/paper/2309.06951/) (2023)
- [RANP: Resource Aware Neuron Pruning at Initialization for 3D CNNs](https://mlsanity.com/paper/2103.08457/) (2021)
- [Shift-Memory Network for Temporal Scene Segmentation](https://mlsanity.com/paper/2202.08399/) (2022)
- [Analysis of Unstructured High-Density Crowded Scenes for Crowd Monitoring](https://mlsanity.com/paper/2408.11836/) (2024)

## Abstract (arXiv)

Convolutional Neural Network (CNN) based image segmentation has made great progress in recent years. However, video object segmentation remains a challenging task due to its high computational complexity. Most of the previous methods employ a two-stream CNN framework to handle spatial and motion features separately. In this paper, we propose an end-to-end encoder-decoder style 3D CNN to aggregate spatial and temporal information simultaneously for video object segmentation. To efficiently process video, we propose 3D separable convolution for the pyramid pooling module and decoder, which dramatically reduces the number of operations while maintaining the performance. Moreover, we also extend our framework to video action segmentation by adding an extra classifier to predict the action label for actors in videos. Extensive experiments on several video datasets demonstrate the superior performance of the proposed approach for action and object segmentation compared to the state-of-the-art.