# SiamMask: A Framework for Fast Online Object Tracking and Segmentation

arXiv: https://arxiv.org/abs/2207.02088 · HTML: https://mlsanity.com/paper/2207.02088/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 73 of its 110 references that are in the map.
- First posted on arXiv 5 Jul 2022.

## Builds on (most cited first, 30 of 73)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [One-Shot Video Object Segmentation](https://mlsanity.com/paper/1611.05198/) (2016)
- [YouTube-VOS: Sequence-to-Sequence Video Object Segmentation](https://mlsanity.com/paper/1809.00461/) (2018)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [Siamese Box Adaptive Network for Visual Tracking](https://mlsanity.com/paper/2003.06761/) (2020)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines](https://mlsanity.com/paper/1911.06188/) (2019)
- [Learning Video Object Segmentation from Static Images](https://mlsanity.com/paper/1612.02646/) (2016)
- [SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking](https://mlsanity.com/paper/1911.07241/) (2019)
- [Detect to Track and Track to Detect](https://mlsanity.com/paper/1710.03958/) (2017)
- [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015)
- [YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video](https://mlsanity.com/paper/1702.00824/) (2017)
- [Online Adaptation of Convolutional Neural Networks for Video Object Segmentation](https://mlsanity.com/paper/1706.09364/) (2017)
- [Efficient Video Object Segmentation via Network Modulation](https://mlsanity.com/paper/1802.01218/) (2018)
- [Learning feed-forward one-shot learners](https://mlsanity.com/paper/1606.05233/) (2016)

## Built on by (most cited first, 13 of 13)

- [Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization](https://mlsanity.com/paper/2211.10528/) (2022)
- [Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Model](https://mlsanity.com/paper/2412.21080/) (2024)
- [CUREE: A Curious Underwater Robot for Ecosystem Exploration](https://mlsanity.com/paper/2303.03943/) (2023)
- [Large-Scale Person Detection and Localization using Overhead Fisheye Cameras](https://mlsanity.com/paper/2307.08252/) (2023)
- [SpikeMOT: Event-based Multi-Object Tracking with Sparse Motion Features](https://mlsanity.com/paper/2309.16987/) (2023)
- [Vision-Based Anti Unmanned Aerial Technology: Opportunities and Challenges](https://mlsanity.com/paper/2507.10006/) (2025)
- [VPTracker: Global Vision-Language Tracking via Visual Prompt](https://mlsanity.com/paper/2512.22799/) (2025)
- [Watching Swarm Dynamics from Above: A Framework for Advanced Object Tracking in Drone Videos](https://mlsanity.com/paper/2406.07680/) (2024)
- [Towards Effective and Efficient Adversarial Defense with Diffusion Models for Robust Visual Tracking](https://mlsanity.com/paper/2506.00325/) (2025)
- [DaDiff: Domain-aware Diffusion Model for Nighttime UAV Tracking](https://mlsanity.com/paper/2410.12270/) (2024)
- [Leveraging Text-to-Image Diffusion Models for Unsupervised Visual Object Tracking](https://mlsanity.com/paper/2605.26933/) (2026)
- [Rotation center identification based on geometric relationships for rotary motion deblurring](https://mlsanity.com/paper/2408.04171/) (2024)
- [Adapting SAM 2 for Visual Object Tracking: 1st Place Solution for MMVPR Challenge Multi-Modal Tracking](https://mlsanity.com/paper/2505.18111/) (2025)

## Abstract (arXiv)

In this paper we introduce SiamMask, a framework to perform both visual object tracking and video object segmentation, in real-time, with the same simple method. We improve the offline training procedure of popular fully-convolutional Siamese approaches by augmenting their losses with a binary segmentation task. Once the offline training is completed, SiamMask only requires a single bounding box for initialization and can simultaneously carry out visual object tracking and segmentation at high frame-rates. Moreover, we show that it is possible to extend the framework to handle multiple object tracking and segmentation by simply re-using the multi-task model in a cascaded fashion. Experimental results show that our approach has high processing efficiency, at around 55 frames per second. It yields real-time state-of-the-art results on visual-object tracking benchmarks, while at the same time demonstrating competitive performance at a high speed for video object segmentation benchmarks.