# Multiple Convolutional Features in Siamese Networks for Object Tracking

arXiv: https://arxiv.org/abs/2103.01222 · HTML: https://mlsanity.com/paper/2103.01222/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015)
- [End-to-end representation learning for Correlation Filter based tracking](https://mlsanity.com/paper/1704.06036/) (2017)
- [Learning to Track at 100 FPS with Deep Regression Networks](https://mlsanity.com/paper/1604.01802/) (2016)
- [A Twofold Siamese Network for Real-Time Object Tracking](https://mlsanity.com/paper/1802.08817/) (2018)
- [Tracking by Instance Detection: A Meta-Learning Approach](https://mlsanity.com/paper/2004.00830/) (2020)
- [Staple: Complementary Learners for Real-Time Tracking](https://mlsanity.com/paper/1512.01355/) (2015)
- ['Skimming-Perusal' Tracking: A Framework for Real-Time and Robust Long-term Tracking](https://mlsanity.com/paper/1909.01840/) (2019)
- [Large Margin Object Tracking with Circulant Feature Maps](https://mlsanity.com/paper/1703.05020/) (2017)
- [Context-aware Deep Feature Compression for High-speed Visual Tracking](https://mlsanity.com/paper/1803.10537/) (2018)
- [Robust Visual Tracking via Hierarchical Convolutional Features](https://mlsanity.com/paper/1707.03816/) (2017)
- [Multi-Branch Siamese Networks with Online Selection for Object Tracking](https://mlsanity.com/paper/1808.07349/) (2018)
- [Coarse-to-Fine Object Tracking Using Deep Features and Correlation Filters](https://mlsanity.com/paper/2012.12784/) (2020)

## Abstract (arXiv)

Siamese trackers demonstrated high performance in object tracking due to their balance between accuracy and speed. Unlike classification-based CNNs, deep similarity networks are specifically designed to address the image similarity problem, and thus are inherently more appropriate for the tracking task. However, Siamese trackers mainly use the last convolutional layers for similarity analysis and target search, which restricts their performance. In this paper, we argue that using a single convolutional layer as feature representation is not an optimal choice in a deep similarity framework. We present a Multiple Features-Siamese Tracker (MFST), a novel tracking algorithm exploiting several hierarchical feature maps for robust tracking. Since convolutional layers provide several abstraction levels in characterizing an object, fusing hierarchical features allows to obtain a richer and more efficient representation of the target. Moreover, we handle the target appearance variations by calibrating the deep features extracted from two different CNN models. Based on this advanced feature representation, our method achieves high tracking accuracy, while outperforming the standard siamese tracker on object tracking benchmarks. The source code and trained models are available at https://github.com/zhenxili96/MFST.