# Correlation-Aware Deep Tracking

arXiv: https://arxiv.org/abs/2203.01666 · HTML: https://mlsanity.com/paper/2203.01666/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 59 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Conditional Positional Encodings for Vision Transformers](https://mlsanity.com/paper/2102.10882/) (2021)
- [Learning Spatio-Temporal Transformer for Visual Tracking](https://mlsanity.com/paper/2103.17154/) (2021)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [Ocean: Object-aware Anchor-free Tracking](https://mlsanity.com/paper/2006.10721/) (2020)
- [Siamese Box Adaptive Network for Visual Tracking](https://mlsanity.com/paper/2003.06761/) (2020)
- [SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines](https://mlsanity.com/paper/1911.06188/) (2019)

## Built on by (most cited first, 30 of 62)

- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [Generalized Relation Modeling for Transformer Tracking](https://mlsanity.com/paper/2303.16580/) (2023)
- [Robust Object Modeling for Visual Tracking](https://mlsanity.com/paper/2308.05140/) (2023)
- [Autoregressive Queries for Adaptive Tracking with Spatio-TemporalTransformers](https://mlsanity.com/paper/2403.10574/) (2024)
- [HIPTrack: Visual Tracking with Historical Prompts](https://mlsanity.com/paper/2311.02072/) (2023)
- [Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance](https://mlsanity.com/paper/2403.05231/) (2024)
- [Towards Unified Token Learning for Vision-Language Tracking](https://mlsanity.com/paper/2308.14103/) (2023)
- [MixFormerV2: Efficient Fully Transformer Tracking](https://mlsanity.com/paper/2305.15896/) (2023)
- [Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Tracking](https://mlsanity.com/paper/2308.06904/) (2023)
- [Towards Universal Modal Tracking with Online Dense Temporal Token Learning](https://mlsanity.com/paper/2507.20177/) (2025)
- [EgoTracks: A Long-term Egocentric Visual Object Tracking Dataset](https://mlsanity.com/paper/2301.03213/) (2023)
- [Target-Aware Tracking with Long-term Context Attention](https://mlsanity.com/paper/2302.13840/) (2023)
- [SPMTrack: Spatio-Temporal Parameter-Efficient Fine-Tuning with Mixture of Experts for Scalable Visual Tracking](https://mlsanity.com/paper/2503.18338/) (2025)
- [Exploring Decoupled Spatio-Temporal Consistency Learning and Self-Prompting Evolution for Self-Supervised Tracking](https://mlsanity.com/paper/2507.21606/) (2025)
- [DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Tasks](https://mlsanity.com/paper/2304.00571/) (2023)
- [Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking](https://mlsanity.com/paper/2504.09228/) (2025)
- [Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking](https://mlsanity.com/paper/2304.14394/) (2023)
- [XTrack: Multimodal Training Boosts RGB-X Video Object Trackers](https://mlsanity.com/paper/2405.17773/) (2024)
- [Learning Spatial-Frequency Transformer for Visual Object Tracking](https://mlsanity.com/paper/2208.08829/) (2022)
- [Synchronize Feature Extracting and Matching: A Single Branch Framework for 3D Object Tracking](https://mlsanity.com/paper/2308.12549/) (2023)
- [LiteTrack: Layer Pruning with Asynchronous Feature Extraction for Lightweight and Efficient Visual Tracking](https://mlsanity.com/paper/2309.09249/) (2023)
- [What You Have is What You Track: Adaptive and Robust Multimodal Tracking](https://mlsanity.com/paper/2507.05899/) (2025)
- [High-Performance Transformer Tracking](https://mlsanity.com/paper/2203.13533/) (2022)
- [Towards Category Unification of 3D Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2401.11204/) (2024)
- [Reading Relevant Feature from Global Representation Memory for Visual Object Tracking](https://mlsanity.com/paper/2402.14392/) (2024)
- [Correlational Image Modeling for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2303.12670/) (2023)
- [Correlation Pyramid Network for 3D Single Object Tracking](https://mlsanity.com/paper/2305.09195/) (2023)
- [Correlation-Embedded Transformer Tracking: A Single-Branch Framework](https://mlsanity.com/paper/2401.12743/) (2024)
- [TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Tracking](https://mlsanity.com/paper/2405.05004/) (2024)

## Abstract (arXiv)

Robustness and discrimination power are two fundamental requirements in visual object tracking. In most tracking paradigms, we find that the features extracted by the popular Siamese-like networks cannot fully discriminatively model the tracked targets and distractor objects, hindering them from simultaneously meeting these two requirements. While most methods focus on designing robust correlation operations, we propose a novel target-dependent feature network inspired by the self-/cross-attention scheme. In contrast to the Siamese-like feature extraction, our network deeply embeds cross-image feature correlation in multiple layers of the feature network. By extensively matching the features of the two images through multiple layers, it is able to suppress non-target features, resulting in instance-varying feature extraction. The output features of the search image can be directly used for predicting target locations without extra correlation step. Moreover, our model can be flexibly pre-trained on abundant unpaired images, leading to notably faster convergence than the existing methods. Extensive experiments show our method achieves the state-of-the-art results while running at real-time. Our feature networks also can be applied to existing tracking pipelines seamlessly to raise the tracking performance. Code will be available.