Fast Online Object Tracking and Segmentation: A Unifying Approach
Not on the map. Why?
Abstract (arXiv)
In this paper we illustrate how to perform both visual object tracking and semi-supervised video object segmentation, in real-time, with a single simple approach. Our method, dubbed SiamMask, improves the offline training procedure of popular fully-convolutional Siamese approaches for object tracking by augmenting their loss with a binary segmentation task. Once trained, SiamMask solely relies on a single bounding box initialisation and operates online, producing class-agnostic object segmentation masks and rotated bounding boxes at 55 frames per second. Despite its simplicity, versatility and fast speed, our strategy allows us to establish a new state of the art among real-time trackers on VOT-2018, while at the same time demonstrating competitive performance and the best speed for the semi-supervised video object segmentation task on DAVIS-2016 and DAVIS-2017. The project website is http://www.robots.ox.ac.uk/~qwang/SiamMask.
Key facts. Cited by 272 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Dec 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 95; 2022: 58; 2023: 53; 2024: 32; 2025: 26; 2026: 8.
272 papers.
Sort:
By area:
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2023Segment Everything Everywhere All at Oncecited by 548
- 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
- 2021Transformer Trackingcited by 412
- 2021Learning Spatio-Temporal Transformer for Visual Trackingcited by 350
- 2022Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Frameworkcited by 330
- 2022MixFormer: End-to-End Tracking with Iterative Mixed Attentioncited by 281
- 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
- 2023Track Anything: Segment Anything Meets Videoscited by 234
- 2021Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Trackingcited by 220
- 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 203
- 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
38 more
- 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192
- 2023Putting the Object Back into Video Object Segmentationcited by 188
- 2021Learning To Count Everythingcited by 150
- 2021Editable Free-viewpoint Video Using a Layered Neural Representationcited by 148
- 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
- 2021Learning Target Candidate Association to Keep Track of What Not to Trackcited by 131
- 2021Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusioncited by 129
- 2021VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flowscited by 129
- 2021Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmarkcited by 126
- 2022Backbone is All Your Need: A Simplified Architecture for Visual Object Trackingcited by 119
- 2022Transforming Model Prediction for Trackingcited by 117
- 2022Towards Grand Unification of Object Trackingcited by 93
- 2021STMTrack: Template-free Visual Tracking with Space-time Memory Networkscited by 91
- 2022Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baselinecited by 91
- 2021LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Searchcited by 82
- 2023Segment Anything Meets Point Trackingcited by 81
- 2024SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Treecited by 80
- 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 77
- 2023Siamese Masked Autoencoderscited by 76
- 2022Transformer Tracking with Cyclic Shifting Window Attentioncited by 75
- 2023Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baselinecited by 73
- 2021HiFT: Hierarchical Feature Transformer for Aerial Trackingcited by 72
- 2023General Object Foundation Model for Images and Videos at Scalecited by 71
- 2022TCTrack: Temporal Contexts for Aerial Trackingcited by 69
- 2021SwiftNet: Real-time Video Object Segmentationcited by 67
- 2021Box-Aware Feature Enhancement for Single Object Tracking on Point Cloudscited by 66
- 2024OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuningcited by 65
- 2021The Multi-Temporal Urban Development SpaceNet Datasetcited by 62
- 2021Do Different Tracking Tasks Require Different Appearance Models?cited by 62
- 2022Correlation-Aware Deep Trackingcited by 62
- 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 62
- 2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
- 20213D Siamese Voxel-to-BEV Tracker for Sparse Point Cloudscited by 54
- 2023Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Trackingcited by 53
- 2023Visual In-Context Promptingcited by 52
- 2021Efficient Visual Tracking with Exemplar Transformerscited by 48
- 2021Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlookcited by 46
- 2023Implicit Temporal Modeling with Learnable Alignment for Video Recognitioncited by 46
50 most cited of 272 All 272 (JSON)
- 2026SWT: Self-Supervised Video Object Segmentation via Sliding, Wavelet and Transportation22 Sep 2026
- 2026Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning2 Aug 2026
- 2026Semantic-Aware Temporal Adaptation for UAV Anti-UAV Tracking29 Jul 2026
- 2026Instance-level Visual Active Tracking with Occlusion-Aware Planning23 Apr 2026
- 2026Interactive Tracking: A Human-in-the-Loop Paradigm with Memory-Augmented Adaptation2 Apr 2026
- 2026MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method20 Feb 2026
- 2026Uncertainty-Guided Inference-Time Depth Adaptation for Transformer-Based Visual Tracking18 Feb 2026
- 2026Few-Shot Video Object Segmentation in X-Ray Angiography Using Local Matching and Spatio-Temporal Consistency Loss2 Jan 2026
- 2025How Far are Modern Trackers from UAV-Anti-UAV? A Million-Scale Benchmark and New Baseline8 Dec 2025
- 2025Video Diffusion Models Excel at Tracking Similar-Looking Objects Without Supervision2 Dec 2025
- 2025SwiTrack: Tri-State Switch for Cross-Modal Object Tracking20 Nov 2025
- 2025UniSOT: A Unified Framework for Multi-Modality Single Object Tracking3 Nov 2025
38 more
- 2025Uirapuru: Timely Video Analytics for High-Resolution Steerable Cameras on Edge Devices1 Sep 2025
- 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes7 Aug 2025
- 2025ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking7 Aug 2025
- 2025Exploring Decoupled Spatio-Temporal Consistency Learning and Self-Prompting Evolution for Self-Supervised Tracking29 Jul 2025
- 2025Towards Universal Modal Tracking with Online Dense Temporal Token Learning27 Jul 2025
- 2025MUVOD: A Novel Multi-view Video Object Segmentation Dataset and A Benchmark for 3D Segmentation10 Jul 2025
- 2025What You Have is What You Track: Adaptive and Robust Multimodal Tracking8 Jul 2025
- 2025Exploiting Lightweight Hierarchical ViT and Dynamic Framework for Efficient Visual Tracking25 Jun 2025
- 2025MVTD: A Benchmark Dataset for Maritime Visual Object Tracking3 Jun 2025
- 2025CSTrack: Enhancing RGB-X Tracking via Compact Spatiotemporal Features26 May 2025
- 2025CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos24 May 2025
- 2025CGTrack: Cascade Gating Network with Hierarchical Feature Aggregation for UAV Tracking9 May 2025
- 2025DCFG: Diverse Cross-Channel Fine-Grained Feature Learning and Progressive Fusion Siamese Tracker for Thermal Infrared Target Tracking19 Apr 2025
- 2025Hyperspectral Adapter for Object Tracking based on Hyperspectral Video28 Mar 2025
- 2025MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking22 Mar 2025
- 2025Dynamic Attention Mechanism in Spatiotemporal Memory Networks for Object Tracking21 Mar 2025
- 2025UncTrack: Reliable Visual Object Tracking with Uncertainty-Aware Prototype Memory Network17 Mar 2025
- 2025Leveraging Motion Information for Better Self-Supervised Video Correspondence Learning15 Mar 2025
- 2025Learning Appearance and Motion Cues for Panoptic Tracking12 Mar 2025
- 2025Find First, Track Next: Decoupling Identification and Propagation in Referring Video Object Segmentation5 Mar 2025
- 2025A Survey on Class-Agnostic Counting: Advancements from Reference-Based to Open-World Text-Guided Approaches31 Jan 2025
- 2025Anomaly Triplet-Net: Progress Recognition Model Using Deep Metric Learning Considering Occlusion for Manual Assembly Work7 Jan 2025
- 2024Stereo Hand-Object Reconstruction for Human-to-Robot Handover10 Dec 2024
- 2024Open-World Drone Active Tracking with Goal-Centered Rewards1 Dec 2024
- 2024Improving Accuracy and Generalization for Efficient Visual Tracking28 Nov 2024
- 2024Interpretable Dynamic Graph Neural Networks for Small Occluded Object Detection and Tracking26 Nov 2024
- 2024VideoOrion: Tokenizing Object Dynamics in Videos25 Nov 2024
- 2024NT-VOT211: A Large-Scale Benchmark for Night-time Visual Object Tracking27 Oct 2024
- 2024Depth Attention for Robust RGB Tracking27 Oct 2024
- 2024SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree21 Oct 2024
- 2024Out-of-Bounding-Box Triggers: A Stealthy Approach to Cheat Object Detectors14 Oct 2024
- 2024DINTR: Tracking via Diffusion-based Interpolation14 Oct 2024
- 2024Improving Visual Object Tracking through Visual Prompting27 Sep 2024
- 2024Convolutional Networks as Extremely Small Foundation Models: Visual Prompting and Theoretical Perspective3 Sep 2024
- 2024EdgeNAT: Transformer for Efficient Edge Detection20 Aug 2024
- 2024Personalized Federated Learning for improving radar based precipitation nowcasting on heterogeneous areas11 Aug 2024
- 2024VoxelTrack: Exploring Voxel Representation for 3D Point Cloud Object Tracking5 Aug 2024
- 2024Progressive Domain Adaptation for Thermal Infrared Object Tracking28 Jul 2024
50 newest of 272 All 272 (JSON)
- 2023Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baselinecited by 73
- 2022Tiny Object Tracking: A Large-scale Dataset and A Baselinecited by 11
- 2026Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruningcited by 0
- 2026Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning2 Aug 2026
- 2023Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline26 Sep 2023
- 2022Tiny Object Tracking: A Large-scale Dataset and A Baseline11 Feb 2022
- 2024Understanding Long Videos with Multimodal Language Modelscited by 25
- 2024Understanding Long Videos with Multimodal Language Models25 Mar 2024
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2023Segment Everything Everywhere All at Oncecited by 548
- 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
- 2021Transformer Trackingcited by 412
- 2021Learning Spatio-Temporal Transformer for Visual Trackingcited by 350
- 2022Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Frameworkcited by 330
- 2022MixFormer: End-to-End Tracking with Iterative Mixed Attentioncited by 281
- 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
- 2023Track Anything: Segment Anything Meets Videoscited by 234
- 2021Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Trackingcited by 220
- 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 203
- 2021Hypercorrelation Squeeze for Few-Shot Segmentationcited by 193
12 most cited of 263 All 263 (JSON)
- 2026SWT: Self-Supervised Video Object Segmentation via Sliding, Wavelet and Transportation22 Sep 2026
- 2026Semantic-Aware Temporal Adaptation for UAV Anti-UAV Tracking29 Jul 2026
- 2026Instance-level Visual Active Tracking with Occlusion-Aware Planning23 Apr 2026
- 2026Interactive Tracking: A Human-in-the-Loop Paradigm with Memory-Augmented Adaptation2 Apr 2026
- 2026MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method20 Feb 2026
- 2026Uncertainty-Guided Inference-Time Depth Adaptation for Transformer-Based Visual Tracking18 Feb 2026
- 2026Few-Shot Video Object Segmentation in X-Ray Angiography Using Local Matching and Spatio-Temporal Consistency Loss2 Jan 2026
- 2025How Far are Modern Trackers from UAV-Anti-UAV? A Million-Scale Benchmark and New Baseline8 Dec 2025
- 2025SwiTrack: Tri-State Switch for Cross-Modal Object Tracking20 Nov 2025
- 2025UniSOT: A Unified Framework for Multi-Modality Single Object Tracking3 Nov 2025
- 2025Uirapuru: Timely Video Analytics for High-Resolution Steerable Cameras on Edge Devices1 Sep 2025
- 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes7 Aug 2025
12 newest of 263 All 263 (JSON)