# Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation

arXiv: https://arxiv.org/abs/2309.11707 · HTML: https://mlsanity.com/paper/2309.11707/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 56 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation](https://mlsanity.com/paper/2106.05210/) (2021)
- [FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation](https://mlsanity.com/paper/1902.09513/) (2019)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [Anisotropic Convolutional Networks for 3D Semantic Scene Completion](https://mlsanity.com/paper/2004.02122/) (2020)
- [STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos](https://mlsanity.com/paper/2003.08429/) (2020)
- [Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications](https://mlsanity.com/paper/2304.05750/) (2023)
- [Video Object Segmentation with Episodic Graph Memory Networks](https://mlsanity.com/paper/2007.07020/) (2020)
- [SegFlow: Joint Learning for Video Object Segmentation and Optical Flow](https://mlsanity.com/paper/1709.06750/) (2017)
- [RVOS: End-to-End Recurrent Network for Video Object Segmentation](https://mlsanity.com/paper/1903.05612/) (2019)
- [Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks](https://mlsanity.com/paper/2001.06807/) (2020)
- [Full-Duplex Strategy for Video Object Segmentation](https://mlsanity.com/paper/2108.03151/) (2021)
- [Learning Fast and Robust Target Models for Video Object Segmentation](https://mlsanity.com/paper/2003.00908/) (2020)
- [SwiftNet: Real-time Video Object Segmentation](https://mlsanity.com/paper/2102.04604/) (2021)
- [Fast Video Object Segmentation using the Global Context Module](https://mlsanity.com/paper/2001.11243/) (2020)
- [Motion-Attentive Transition for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2003.04253/) (2020)
- [Making a Case for 3D Convolutions for Object Segmentation in Videos](https://mlsanity.com/paper/2008.11516/) (2020)
- [Anchor Diffusion for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/1910.10895/) (2019)
- [Local-Global Context Aware Transformer for Language-Guided Video Segmentation](https://mlsanity.com/paper/2203.09773/) (2022)
- [Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2209.03138/) (2022)
- [UnOVOST: Unsupervised Offline Video Object Segmentation and Tracking](https://mlsanity.com/paper/2001.05425/) (2020)
- [EM-driven unsupervised learning for efficient motion segmentation](https://mlsanity.com/paper/2201.02074/) (2022)

## Built on by (most cited first, 5 of 5)

- [Adversarial Attacks on Video Object Segmentation with Hard Region Discovery](https://mlsanity.com/paper/2309.13857/) (2023)
- [Learning Motion and Temporal Cues for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2501.07806/) (2025)
- [GISE-TTT:A Framework for Global InformationSegmentation and Enhancement](https://mlsanity.com/paper/2504.00879/) (2025)
- [Unleashing the Power of Motion and Depth: A Selective Fusion Strategy for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2507.21857/) (2025)
- [M4-SAM: Multi-Modal Mixture-of-Experts with Memory-Augmented SAM for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2605.11760/) (2026)

## Abstract (arXiv)

Unsupervised Video Object Segmentation (VOS) aims at identifying the contours of primary foreground objects in videos without any prior knowledge. However, previous methods do not fully use spatial-temporal context and fail to tackle this challenging task in real-time. This motivates us to develop an efficient Long-Short Temporal Attention network (termed LSTA) for unsupervised VOS task from a holistic view. Specifically, LSTA consists of two dominant modules, i.e., Long Temporal Memory and Short Temporal Attention. The former captures the long-term global pixel relations of the past frames and the current frame, which models constantly present objects by encoding appearance pattern. Meanwhile, the latter reveals the short-term local pixel relations of one nearby frame and the current frame, which models moving objects by encoding motion pattern. To speedup the inference, the efficient projection and the locality-based sliding window are adopted to achieve nearly linear time complexity for the two light modules, respectively. Extensive empirical studies on several benchmarks have demonstrated promising performances of the proposed method with high efficiency.