# Investigation of Frame Differences as Motion Cues for Video Object Segmentation

arXiv: https://arxiv.org/abs/2503.09132 · HTML: https://mlsanity.com/paper/2503.09132/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 15 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [SegFlow: Joint Learning for Video Object Segmentation and Optical Flow](https://mlsanity.com/paper/1709.06750/) (2017)
- [FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos](https://mlsanity.com/paper/1701.05384/) (2017)
- [Learning Video Object Segmentation with Visual Memory](https://mlsanity.com/paper/1704.05737/) (2017)
- [Motion-Attentive Transition for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2003.04253/) (2020)
- [Unsupervised Video Object Segmentation using Motion Saliency-Guided Spatio-Temporal Propagation](https://mlsanity.com/paper/1809.01125/) (2018)
- [Optical Flow Estimation in the Deep Learning Age](https://mlsanity.com/paper/2004.02853/) (2020)

## Abstract (arXiv)

Automatic Video Object Segmentation (AVOS) refers to the task of autonomously segmenting target objects in video sequences without relying on human-provided annotations in the first frames. In AVOS, the use of motion information is crucial, with optical flow being a commonly employed method for capturing motion cues. However, the computation of optical flow is resource-intensive, making it unsuitable for real-time applications, especially on edge devices with limited computational resources. In this study, we propose using frame differences as an alternative to optical flow for motion cue extraction. We developed an extended U-Net-like AVOS model that takes a frame on which segmentation is performed and a frame difference as inputs, and outputs an estimated segmentation map. Our experimental results demonstrate that the proposed model achieves performance comparable to the model with optical flow as an input, particularly when applied to videos captured by stationary cameras. Our results suggest the usefulness of employing frame differences as motion cues in cases with limited computational resources.