# SwiftNet: Real-time Video Object Segmentation

arXiv: https://arxiv.org/abs/2102.04604 · HTML: https://mlsanity.com/paper/2102.04604/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 46 references that are in the map.
- First posted on arXiv 9 Feb 2021.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [One-Shot Video Object Segmentation](https://mlsanity.com/paper/1611.05198/) (2016)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentation](https://mlsanity.com/paper/1902.09513/) (2019)
- [Learning Video Object Segmentation from Static Images](https://mlsanity.com/paper/1612.02646/) (2016)
- [Collaborative Video Object Segmentation by Foreground-Background Integration](https://mlsanity.com/paper/2003.08333/) (2020)
- [See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks](https://mlsanity.com/paper/2001.06810/) (2020)
- [Online Adaptation of Convolutional Neural Networks for Video Object Segmentation](https://mlsanity.com/paper/1706.09364/) (2017)
- [Efficient Video Object Segmentation via Network Modulation](https://mlsanity.com/paper/1802.01218/) (2018)
- [Video Object Segmentation Without Temporal Information](https://mlsanity.com/paper/1709.06031/) (2017)
- [RVOS: End-to-End Recurrent Network for Video Object Segmentation](https://mlsanity.com/paper/1903.05612/) (2019)
- [PReMVOS: Proposal-generation, Refinement and Merging for Video Object Segmentation](https://mlsanity.com/paper/1807.09190/) (2018)
- [FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videos](https://mlsanity.com/paper/1701.05384/) (2017)
- [Learning Fast and Robust Target Models for Video Object Segmentation](https://mlsanity.com/paper/2003.00908/) (2020)
- [Learning Video Object Segmentation with Visual Memory](https://mlsanity.com/paper/1704.05737/) (2017)
- [A Generative Appearance Model for End-to-end Video Object Segmentation](https://mlsanity.com/paper/1811.11611/) (2018)
- [Fast Video Object Segmentation using the Global Context Module](https://mlsanity.com/paper/2001.11243/) (2020)
- [RANet: Ranking Attention Network for Fast Video Object Segmentation](https://mlsanity.com/paper/1908.06647/) (2019)
- [Fast and Accurate Online Video Object Segmentation via Tracking Parts](https://mlsanity.com/paper/1806.02323/) (2018)
- [Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagation](https://mlsanity.com/paper/1803.04242/) (2018)
- [A Transductive Approach for Video Object Segmentation](https://mlsanity.com/paper/2004.07193/) (2020)
- [State-Aware Tracker for Real-Time Video Object Segmentation](https://mlsanity.com/paper/2003.00482/) (2020)
- [Video Object Segmentation and Tracking: A Survey](https://mlsanity.com/paper/1904.09172/) (2019)
- [Anchor Diffusion for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/1910.10895/) (2019)

## Built on by (most cited first, 30 of 67)

- [XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model](https://mlsanity.com/paper/2207.07115/) (2022)
- [Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation](https://mlsanity.com/paper/2106.05210/) (2021)
- [Putting the Object Back into Video Object Segmentation](https://mlsanity.com/paper/2310.12982/) (2023)
- [A Survey on Deep Learning Technique for Video Segmentation](https://mlsanity.com/paper/2107.01153/) (2021)
- [LVOS: A Benchmark for Long-term Video Object Segmentation](https://mlsanity.com/paper/2211.10181/) (2022)
- [XMem++: Production-level Video Segmentation From Few Annotated Frames](https://mlsanity.com/paper/2307.15958/) (2023)
- [Recurrent Dynamic Embedding for Video Object Segmentation](https://mlsanity.com/paper/2205.03761/) (2022)
- [Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentation](https://mlsanity.com/paper/2206.03789/) (2022)
- [LVOS: A Benchmark for Large-scale Long-term Video Object Segmentation](https://mlsanity.com/paper/2404.19326/) (2024)
- [Reliable Propagation-Correction Modulation for Video Object Segmentation](https://mlsanity.com/paper/2112.02853/) (2021)
- [Learning Quality-aware Dynamic Memory for Video Object Segmentation](https://mlsanity.com/paper/2207.07922/) (2022)
- [Look Before You Match: Instance Understanding Matters in Video Object Segmentation](https://mlsanity.com/paper/2212.06826/) (2022)
- [SWEM: Towards Real-Time Video Object Segmentation with Sequential Weighted Expectation-Maximization](https://mlsanity.com/paper/2208.10128/) (2022)
- [MatAnyone: Stable Video Matting with Consistent Memory Propagation](https://mlsanity.com/paper/2501.14677/) (2025)
- [Global Spectral Filter Memory Network for Video Object Segmentation](https://mlsanity.com/paper/2210.05567/) (2022)
- [Boosting Video Object Segmentation via Space-time Correspondence Learning](https://mlsanity.com/paper/2304.06211/) (2023)
- [Region Aware Video Object Segmentation with Deep Motion Modeling](https://mlsanity.com/paper/2207.10258/) (2022)
- [LiDAR-Camera Panoptic Segmentation via Geometry-Consistent and Semantic-Aware Alignment](https://mlsanity.com/paper/2308.01686/) (2023)
- [PanoVOS: Bridging Non-panoramic and Panoramic Views with Transformer for Video Segmentation](https://mlsanity.com/paper/2309.12303/) (2023)
- [Tackling Background Distraction in Video Object Segmentation](https://mlsanity.com/paper/2207.06953/) (2022)
- [MobileVOS: Real-Time Video Object Segmentation Contrastive Learning meets Knowledge Distillation](https://mlsanity.com/paper/2303.07815/) (2023)
- [TransVOS: Video Object Segmentation with Transformers](https://mlsanity.com/paper/2106.00588/) (2021)
- [ARKitTrack: A New Diverse Dataset for Tracking Using Mobile RGB-D Data](https://mlsanity.com/paper/2303.13885/) (2023)
- [PyramidMamba: Rethinking Pyramid Feature Fusion with Selective Space State Model for Semantic Segmentation of Remote Sensing Imagery](https://mlsanity.com/paper/2406.10828/) (2024)
- [Siamese Network with Interactive Transformer for Video Object Segmentation](https://mlsanity.com/paper/2112.13983/) (2021)
- [Accelerating Video Object Segmentation with Compressed Video](https://mlsanity.com/paper/2107.12192/) (2021)
- [Nav-R2 Dual-Relation Reasoning for Generalizable Open-Vocabulary Object-Goal Navigation](https://mlsanity.com/paper/2512.02400/) (2025)
- [Panoramic Video Salient Object Detection with Ambisonic Audio Guidance](https://mlsanity.com/paper/2211.14419/) (2022)
- [Co-attention Propagation Network for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2304.03910/) (2023)
- [Monocular 3D lane detection for Autonomous Driving: Recent Achievements, Challenges, and Outlooks](https://mlsanity.com/paper/2404.06860/) (2024)

## Abstract (arXiv)

In this work we present SwiftNet for real-time semisupervised video object segmentation (one-shot VOS), which reports 77.8% J &F and 70 FPS on DAVIS 2017 validation dataset, leading all present solutions in overall accuracy and speed performance. We achieve this by elaborately compressing spatiotemporal redundancy in matching-based VOS via Pixel-Adaptive Memory (PAM). Temporally, PAM adaptively triggers memory updates on frames where objects display noteworthy inter-frame variations. Spatially, PAM selectively performs memory update and match on dynamic pixels while ignoring the static ones, significantly reducing redundant computations wasted on segmentation-irrelevant pixels. To promote efficient reference encoding, light-aggregation encoder is also introduced in SwiftNet deploying reversed sub-pixel. We hope SwiftNet could set a strong and efficient baseline for real-time VOS and facilitate its application in mobile vision. The source code of SwiftNet can be found at https://github.com/haochenheheda/SwiftNet.