# Deeper and Wider Siamese Networks for Real-Time Visual Tracking

arXiv: https://arxiv.org/abs/1901.01660 · HTML: https://mlsanity.com/paper/1901.01660/

## Key facts

- Cited by 156 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2019.

## Built on by (most cited first, 30 of 156)

- [Learning Spatio-Temporal Transformer for Visual Tracking](https://mlsanity.com/paper/2103.17154/) (2021)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [AiATrack: Attention in Attention for Transformer Visual Tracking](https://mlsanity.com/paper/2207.09603/) (2022)
- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark](https://mlsanity.com/paper/2103.16746/) (2021)
- [SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory](https://mlsanity.com/paper/2411.11922/) (2024)
- [Generalized Relation Modeling for Transformer Tracking](https://mlsanity.com/paper/2303.16580/) (2023)
- [STMTrack: Template-free Visual Tracking with Space-time Memory Networks](https://mlsanity.com/paper/2104.00324/) (2021)
- [Object Tracking by Jointly Exploiting Frame and Event Domain](https://mlsanity.com/paper/2109.09052/) (2021)
- [LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search](https://mlsanity.com/paper/2104.14545/) (2021)
- [Transformer Tracking with Cyclic Shifting Window Attention](https://mlsanity.com/paper/2205.03806/) (2022)
- [TCTrack: Temporal Contexts for Aerial Tracking](https://mlsanity.com/paper/2203.01885/) (2022)
- [Learn to Match: Automatic Matching Network Design for Visual Tracking](https://mlsanity.com/paper/2108.00803/) (2021)
- [Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2108.04728/) (2021)
- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [SiamMOT: Siamese Multi-Object Tracking](https://mlsanity.com/paper/2105.11595/) (2021)
- [Correlation-Aware Deep Tracking](https://mlsanity.com/paper/2203.01666/) (2022)
- [GuardT2I: Defending Text-to-Image Models from Adversarial Prompts](https://mlsanity.com/paper/2403.01446/) (2024)
- [MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2508.05630/) (2025)
- [Accelerating DETR Convergence via Semantic-Aligned Matching](https://mlsanity.com/paper/2203.06883/) (2022)
- [Unsupervised Domain Adaptation for Nighttime Aerial Tracking](https://mlsanity.com/paper/2203.10541/) (2022)
- [Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Tracking](https://mlsanity.com/paper/2308.06904/) (2023)
- [Divert More Attention to Vision-Language Tracking](https://mlsanity.com/paper/2207.01076/) (2022)
- [Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook](https://mlsanity.com/paper/2112.02838/) (2021)
- [SiamAPN++: Siamese Attentional Aggregation Network for Real-Time UAV Tracking](https://mlsanity.com/paper/2106.08816/) (2021)
- [FEAR: Fast, Efficient, Accurate and Robust Visual Tracker](https://mlsanity.com/paper/2112.07957/) (2021)
- [Exploring Enhanced Contextual Information for Video-Level Object Tracking](https://mlsanity.com/paper/2412.11023/) (2024)
- [DISCO: accurate Discrete Scale Convolutions](https://mlsanity.com/paper/2106.02733/) (2021)
- [MFGNet: Dynamic Modality-Aware Filter Generation for RGB-T Tracking](https://mlsanity.com/paper/2107.10433/) (2021)
- [Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking](https://mlsanity.com/paper/2101.08466/) (2021)

## Abstract (arXiv)

Siamese networks have drawn great attention in visual tracking because of their balanced accuracy and speed. However, the backbone networks used in Siamese trackers are relatively shallow, such as AlexNet [18], which does not fully take advantage of the capability of modern deep neural networks. In this paper, we investigate how to leverage deeper and wider convolutional neural networks to enhance tracking robustness and accuracy. We observe that direct replacement of backbones with existing powerful architectures, such as ResNet [14] and Inception [33], does not bring improvements. The main reasons are that 1)large increases in the receptive field of neurons lead to reduced feature discriminability and localization precision; and 2) the network padding for convolutions induces a positional bias in learning. To address these issues, we propose new residual modules to eliminate the negative impact of padding, and further design new architectures using these modules with controlled receptive field size and network stride. The designed architectures are lightweight and guarantee real-time tracking speed when applied to SiamFC [2] and SiamRPN [20]. Experiments show that solely due to the proposed network architectures, our SiamFC+ and SiamRPN+ obtain up to 9.8%/5.7% (AUC), 23.3%/8.8% (EAO) and 24.4%/25.0% (EAO) relative improvements over the original versions [2, 20] on the OTB-15, VOT-16 and VOT-17 datasets, respectively.