# Siamese Instance Search for Tracking

arXiv: https://arxiv.org/abs/1605.05863 · HTML: https://mlsanity.com/paper/1605.05863/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2016.

## Built on by (most cited first, 30 of 60)

- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search](https://mlsanity.com/paper/2104.14545/) (2021)
- [Monocular Quasi-Dense 3D Object Tracking](https://mlsanity.com/paper/2103.07351/) (2021)
- [Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline](https://mlsanity.com/paper/2309.14611/) (2023)
- [PTTR: Relational 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2112.02857/) (2021)
- [Unsupervised Domain Adaptation for Nighttime Aerial Tracking](https://mlsanity.com/paper/2203.10541/) (2022)
- [Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Tracking](https://mlsanity.com/paper/2308.06904/) (2023)
- [RelationTrack: Relation-aware Multiple Object Tracking with Decoupled Representation](https://mlsanity.com/paper/2105.04322/) (2021)
- [Divert More Attention to Vision-Language Tracking](https://mlsanity.com/paper/2207.01076/) (2022)
- [3D Siamese Transformer Network for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2207.11995/) (2022)
- [FEAR: Fast, Efficient, Accurate and Robust Visual Tracker](https://mlsanity.com/paper/2112.07957/) (2021)
- [Learning to Associate Every Segment for Video Panoptic Segmentation](https://mlsanity.com/paper/2106.09453/) (2021)
- [Learning Target-aware Representation for Visual Tracking via Informative Interactions](https://mlsanity.com/paper/2201.02526/) (2022)
- [Mamba-FETrack: Frame-Event Tracking via State Space Model](https://mlsanity.com/paper/2404.18174/) (2024)
- [Active Learning for Deep Visual Tracking](https://mlsanity.com/paper/2110.13259/) (2021)
- [Exploring Point-BEV Fusion for 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2208.05216/) (2022)
- [Semi-Supervised Visual Tracking of Marine Animals using Autonomous Underwater Vehicles](https://mlsanity.com/paper/2302.07344/) (2023)
- [LiteTrack: Layer Pruning with Asynchronous Feature Extraction for Lightweight and Efficient Visual Tracking](https://mlsanity.com/paper/2309.09249/) (2023)
- [High-Performance Transformer Tracking](https://mlsanity.com/paper/2203.13533/) (2022)
- [SiamMask: A Framework for Fast Online Object Tracking and Segmentation](https://mlsanity.com/paper/2207.02088/) (2022)
- [Divert More Attention to Vision-Language Object Tracking](https://mlsanity.com/paper/2307.10046/) (2023)
- [T-MAE: Temporal Masked Autoencoders for Point Cloud Representation Learning](https://mlsanity.com/paper/2312.10217/) (2023)
- [Adversarial Attacks on Video Object Segmentation with Hard Region Discovery](https://mlsanity.com/paper/2309.13857/) (2023)
- [Temporal Unlearnable Examples: Preventing Personal Video Data from Unauthorized Exploitation by Object Tracking](https://mlsanity.com/paper/2507.07483/) (2025)
- [ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking](https://mlsanity.com/paper/2508.05221/) (2025)
- [Adaptive Hierarchical Similarity Metric Learning with Noisy Labels](https://mlsanity.com/paper/2111.00006/) (2021)
- [Modeling Continuous Motion for 3D Point Cloud Object Tracking](https://mlsanity.com/paper/2303.07605/) (2023)
- [Few-Shot Video Object Detection](https://mlsanity.com/paper/2104.14805/) (2021)
- [Semantic-Aligned Matching for Enhanced DETR Convergence and Multi-Scale Feature Fusion](https://mlsanity.com/paper/2207.14172/) (2022)

## Abstract (arXiv)

In this paper we present a tracker, which is radically different from state-of-the-art trackers: we apply no model updating, no occlusion detection, no combination of trackers, no geometric matching, and still deliver state-of-the-art tracking performance, as demonstrated on the popular online tracking benchmark (OTB) and six very challenging YouTube videos. The presented tracker simply matches the initial patch of the target in the first frame with candidates in a new frame and returns the most similar patch by a learned matching function. The strength of the matching function comes from being extensively trained generically, i.e., without any data of the target, using a Siamese deep neural network, which we design for tracking. Once learned, the matching function is used as is, without any adapting, to track previously unseen targets. It turns out that the learned matching function is so powerful that a simple tracker built upon it, coined Siamese INstance search Tracker, SINT, which only uses the original observation of the target from the first frame, suffices to reach state-of-the-art performance. Further, we show the proposed tracker even allows for target re-identification after the target was absent for a complete video shot.