# Motion-aware Memory Network for Fast Video Salient Object Detection

arXiv: https://arxiv.org/abs/2208.00946 · HTML: https://mlsanity.com/paper/2208.00946/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 65 references that are in the map.
- First posted on arXiv 1 Aug 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Structure-measure: A New Way to Evaluate Foreground Maps](https://mlsanity.com/paper/1708.00786/) (2017)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Key-Value Memory Networks for Directly Reading Documents](https://mlsanity.com/paper/1606.03126/) (2016)
- [A Simple Pooling-Based Design for Real-Time Salient Object Detection](https://mlsanity.com/paper/1904.09569/) (2019)
- [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016)
- [Ask Me Anything: Dynamic Memory Networks for Natural Language Processing](https://mlsanity.com/paper/1506.07285/) (2015)
- [Kernelized Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2007.08270/) (2020)
- [Label Decoupling Framework for Salient Object Detection](https://mlsanity.com/paper/2008.11048/) (2020)
- [Global Context-Aware Progressive Aggregation Network for Salient Object Detection](https://mlsanity.com/paper/2003.00651/) (2020)
- [Efficient Regional Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2103.12934/) (2021)
- [Full-Duplex Strategy for Video Object Segmentation](https://mlsanity.com/paper/2108.03151/) (2021)
- [Motion Guided Attention for Video Salient Object Detection](https://mlsanity.com/paper/1909.07061/) (2019)
- [Learning Position and Target Consistency for Memory-based Video Object Segmentation](https://mlsanity.com/paper/2104.04329/) (2021)
- [Learning Dynamic Memory Networks for Object Tracking](https://mlsanity.com/paper/1803.07268/) (2018)
- [Semi-Supervised Video Salient Object Detection Using Pseudo-Labels](https://mlsanity.com/paper/1908.04051/) (2019)
- [Selectivity or Invariance: Boundary-aware Salient Object Detection](https://mlsanity.com/paper/1812.10066/) (2018)
- [Video Salient Object Detection via Fully Convolutional Networks](https://mlsanity.com/paper/1702.00871/) (2017)
- [Attend to You: Personalized Image Captioning with Context Sequence Memory Networks](https://mlsanity.com/paper/1704.06485/) (2017)
- [Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton](https://mlsanity.com/paper/2004.08595/) (2020)
- [Weakly Supervised Video Salient Object Detection](https://mlsanity.com/paper/2104.02391/) (2021)
- [TENet: Triple Excitation Network for Video Salient Object Detection](https://mlsanity.com/paper/2007.09943/) (2020)
- [Person Re-identification by Saliency Learning](https://mlsanity.com/paper/1412.1908/) (2014)
- [A Read-Write Memory Network for Movie Story Understanding](https://mlsanity.com/paper/1709.09345/) (2017)
- [Exploring Rich and Efficient Spatial Temporal Interactions for Real Time Video Salient Object Detection](https://mlsanity.com/paper/2008.02973/) (2020)
- [Video Salient Object Detection Using Spatiotemporal Deep Features](https://mlsanity.com/paper/1708.01447/) (2017)

## Built on by (most cited first, 12 of 12)

- [Explicit Motion Handling and Interactive Prompting for Video Camouflaged Object Detection](https://mlsanity.com/paper/2403.01968/) (2024)
- [Inspiring the Next Generation of Segment Anything Models: Comprehensively Evaluate SAM and SAM 2 with Diverse Prompts Towards Context-Dependent Concepts under Different Scenes](https://mlsanity.com/paper/2412.01240/) (2024)
- [SOTA: Spike-Navigated Optimal TrAnsport Saliency Region Detection in Composite-bias Videos](https://mlsanity.com/paper/2505.00394/) (2025)
- [CamoSAM2: SAM2-oriented Prompt Auto-Refinement for Video Camouflaged Object Detection](https://mlsanity.com/paper/2504.00375/) (2025)
- [GAPNet: A Lightweight Framework for Image and Video Salient Object Detection via Granularity-Aware Paradigm](https://mlsanity.com/paper/2508.07585/) (2025)
- [A Spatial-Temporal Dual-Mode Mixed Flow Network for Panoramic Video Salient Object Detection](https://mlsanity.com/paper/2310.09016/) (2023)
- [Deformable Feature Alignment and Refinement for Moving Infrared Dim-small Target Detection](https://mlsanity.com/paper/2407.07289/) (2024)
- [Balancing long- and short-term dynamics for the modeling of saliency in videos](https://mlsanity.com/paper/2504.05913/) (2025)
- [Quaternion-Based Robust PCA for Efficient Moving Target Detection and Background Recovery in Color Videos](https://mlsanity.com/paper/2507.19730/) (2025)
- [Unleashing the Power of Motion and Depth: A Selective Fusion Strategy for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2507.21857/) (2025)
- [DGSSM: Diffusion guided state-space models for multimodal salient object detection](https://mlsanity.com/paper/2604.17585/) (2026)
- [Liquid Fusion of Heterogeneous Representations Towards General Salient Object Detection](https://mlsanity.com/paper/2606.26849/) (2026)

## Abstract (arXiv)

Previous methods based on 3DCNN, convLSTM, or optical flow have achieved great success in video salient object detection (VSOD). However, they still suffer from high computational costs or poor quality of the generated saliency maps. To solve these problems, we design a space-time memory (STM)-based network, which extracts useful temporal information of the current frame from adjacent frames as the temporal branch of VSOD. Furthermore, previous methods only considered single-frame prediction without temporal association. As a result, the model may not focus on the temporal information sufficiently. Thus, we initially introduce object motion prediction between inter-frame into VSOD. Our model follows standard encoder--decoder architecture. In the encoding stage, we generate high-level temporal features by using high-level features from the current and its adjacent frames. This approach is more efficient than the optical flow-based methods. In the decoding stage, we propose an effective fusion strategy for spatial and temporal branches. The semantic information of the high-level features is used to fuse the object details in the low-level features, and then the spatiotemporal features are obtained step by step to reconstruct the saliency maps. Moreover, inspired by the boundary supervision commonly used in image salient object detection (ISOD), we design a motion-aware loss for predicting object boundary motion and simultaneously perform multitask learning for VSOD and object motion prediction, which can further facilitate the model to extract spatiotemporal features accurately and maintain the object integrity. Extensive experiments on several datasets demonstrated the effectiveness of our method and can achieve state-of-the-art metrics on some datasets. The proposed model does not require optical flow or other preprocessing, and can reach a speed of nearly 100 FPS during inference.