# A Twofold Siamese Network for Real-Time Object Tracking

arXiv: https://arxiv.org/abs/1802.08817 · HTML: https://mlsanity.com/paper/1802.08817/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2018.

## Built on by (most cited first, 30 of 62)

- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams](https://mlsanity.com/paper/2406.08085/) (2024)
- [Flash-VStream: Efficient Real-Time Understanding for Long Video Streams](https://mlsanity.com/paper/2506.23825/) (2025)
- [SiamMOT: Siamese Multi-Object Tracking](https://mlsanity.com/paper/2105.11595/) (2021)
- [GuardT2I: Defending Text-to-Image Models from Adversarial Prompts](https://mlsanity.com/paper/2403.01446/) (2024)
- [Accelerating DETR Convergence via Semantic-Aligned Matching](https://mlsanity.com/paper/2203.06883/) (2022)
- [3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds](https://mlsanity.com/paper/2111.04426/) (2021)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook](https://mlsanity.com/paper/2112.02838/) (2021)
- [Synchronize Feature Extracting and Matching: A Single Branch Framework for 3D Object Tracking](https://mlsanity.com/paper/2308.12549/) (2023)
- [Learning Tracking Representations via Dual-Branch Fully Transformer Networks](https://mlsanity.com/paper/2112.02571/) (2021)
- [High-Performance Transformer Tracking](https://mlsanity.com/paper/2203.13533/) (2022)
- [The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey](https://mlsanity.com/paper/2305.13854/) (2023)
- [Generative AI for Unmanned Vehicle Swarms: Challenges, Applications and Opportunities](https://mlsanity.com/paper/2402.18062/) (2024)
- [SiamMask: A Framework for Fast Online Object Tracking and Segmentation](https://mlsanity.com/paper/2207.02088/) (2022)
- [Siamese Object Tracking for Unmanned Aerial Vehicle: A Review and Comprehensive Analysis](https://mlsanity.com/paper/2205.04281/) (2022)
- [T-MAE: Temporal Masked Autoencoders for Point Cloud Representation Learning](https://mlsanity.com/paper/2312.10217/) (2023)
- [Correlation Pyramid Network for 3D Single Object Tracking](https://mlsanity.com/paper/2305.09195/) (2023)
- [Speak While Watching: Unleashing TRUE Real-Time Video Understanding Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2601.06843/) (2026)
- [Learning Localization-aware Target Confidence for Siamese Visual Tracking](https://mlsanity.com/paper/2204.14093/) (2022)
- [Vision-based Human Fall Detection Systems using Deep Learning: A Review](https://mlsanity.com/paper/2207.10952/) (2022)
- [Semantic-Aligned Matching for Enhanced DETR Convergence and Multi-Scale Feature Fusion](https://mlsanity.com/paper/2207.14172/) (2022)
- [Multi-domain Collaborative Feature Representation for Robust Visual Object Tracking](https://mlsanity.com/paper/2108.04521/) (2021)
- [UHP-SOT: An Unsupervised High-Performance Single Object Tracker](https://mlsanity.com/paper/2110.01812/) (2021)
- [FlakyFix: Using Large Language Models for Predicting Flaky Test Fix Categories and Test Code Repair](https://mlsanity.com/paper/2307.00012/) (2023)
- [Ad2Attack: Adaptive Adversarial Attack on Real-Time UAV Tracking](https://mlsanity.com/paper/2203.01516/) (2022)
- [Adaptive Siamese Tracking with a Compact Latent Network](https://mlsanity.com/paper/2302.00930/) (2023)
- [Distilling Channels for Efficient Deep Tracking](https://mlsanity.com/paper/2409.11785/) (2024)
- [SiamEvent: Event-based Object Tracking via Edge-aware Similarity Learning with Siamese Networks](https://mlsanity.com/paper/2109.13456/) (2021)
- [A Discriminative Single-Shot Segmentation Network for Visual Object Tracking](https://mlsanity.com/paper/2112.11846/) (2021)

## Abstract (arXiv)

Observing that Semantic features learned in an image classification task and Appearance features learned in a similarity matching task complement each other, we build a twofold Siamese network, named SA-Siam, for real-time object tracking. SA-Siam is composed of a semantic branch and an appearance branch. Each branch is a similarity-learning Siamese network. An important design choice in SA-Siam is to separately train the two branches to keep the heterogeneity of the two types of features. In addition, we propose a channel attention mechanism for the semantic branch. Channel-wise weights are computed according to the channel activations around the target position. While the inherited architecture from SiamFC citeSiamFC allows our tracker to operate beyond real-time, the twofold design and the attention mechanism significantly improve the tracking performance. The proposed SA-Siam outperforms all other real-time trackers by a large margin on OTB-2013/50/100 benchmarks.