# Learning to Fuse Asymmetric Feature Maps in Siamese Trackers

arXiv: https://arxiv.org/abs/2012.02776 · HTML: https://mlsanity.com/paper/2012.02776/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2020.

## Built on by (most cited first, 18 of 18)

- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [ODTrack: Online Dense Temporal Token Learning for Visual Tracking](https://mlsanity.com/paper/2401.01686/) (2024)
- [SwinTrack: A Simple and Strong Baseline for Transformer Tracking](https://mlsanity.com/paper/2112.00995/) (2021)
- [Towards Universal Modal Tracking with Online Dense Temporal Token Learning](https://mlsanity.com/paper/2507.20177/) (2025)
- [Real-time 3D Single Object Tracking with Transformer](https://mlsanity.com/paper/2209.00860/) (2022)
- [Active Learning for Deep Visual Tracking](https://mlsanity.com/paper/2110.13259/) (2021)
- [Unsupervised Learning of Accurate Siamese Tracking](https://mlsanity.com/paper/2204.01475/) (2022)
- [High-Performance Transformer Tracking](https://mlsanity.com/paper/2203.13533/) (2022)
- [Ranking-Based Siamese Visual Tracking](https://mlsanity.com/paper/2205.11761/) (2022)
- [Lightweight Full-Convolutional Siamese Tracker](https://mlsanity.com/paper/2310.05392/) (2023)
- [Learning Localization-aware Target Confidence for Siamese Visual Tracking](https://mlsanity.com/paper/2204.14093/) (2022)
- [Underwater Object Tracker: UOSTrack for Marine Organism Grasping of Underwater Vehicles](https://mlsanity.com/paper/2301.01482/) (2023)
- [RGB-Sonar Tracking Benchmark and Spatial Cross-Attention Transformer Tracker](https://mlsanity.com/paper/2406.07189/) (2024)
- [OST: Efficient One-stream Network for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2210.08518/) (2022)
- [Rethinking Clustering-Based Pseudo-Labeling for Unsupervised Meta-Learning](https://mlsanity.com/paper/2209.13635/) (2022)
- [NT-VOT211: A Large-Scale Benchmark for Night-time Visual Object Tracking](https://mlsanity.com/paper/2410.20421/) (2024)
- [Leveraging Text-to-Image Diffusion Models for Unsupervised Visual Object Tracking](https://mlsanity.com/paper/2605.26933/) (2026)
- [Search Multilayer Perceptron-Based Fusion for Efficient and Accurate Siamese Tracking](https://mlsanity.com/paper/2603.01706/) (2026)

## Abstract (arXiv)

Recently, Siamese-based trackers have achieved promising performance in visual tracking. Most recent Siamese-based trackers typically employ a depth-wise cross-correlation (DW-XCorr) to obtain multi-channel correlation information from the two feature maps (target and search region). However, DW-XCorr has several limitations within Siamese-based tracking: it can easily be fooled by distractors, has fewer activated channels, and provides weak discrimination of object boundaries. Further, DW-XCorr is a handcrafted parameter-free module and cannot fully benefit from offline learning on large-scale data. We propose a learnable module, called the asymmetric convolution (ACM), which learns to better capture the semantic correlation information in offline training on large-scale data. Different from DW-XCorr and its predecessor(XCorr), which regard a single feature map as the convolution kernel, our ACM decomposes the convolution operation on a concatenated feature map into two mathematically equivalent operations, thereby avoiding the need for the feature maps to be of the same size (width and height)during concatenation. Our ACM can incorporate useful prior information, such as bounding-box size, with standard visual features. Furthermore, ACM can easily be integrated into existing Siamese trackers based on DW-XCorror XCorr. To demonstrate its generalization ability, we integrate ACM into three representative trackers: SiamFC, SiamRPN++, and SiamBAN. Our experiments reveal the benefits of the proposed ACM, which outperforms existing methods on six tracking benchmarks. On the LaSOT test set, our ACM-based tracker obtains a significant improvement of 5.8% in terms of success (AUC), over the baseline.