# Tracking by Instance Detection: A Meta-Learning Approach

arXiv: https://arxiv.org/abs/2004.00830 · HTML: https://mlsanity.com/paper/2004.00830/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2020.

## Built on by (most cited first, 30 of 61)

- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Learning Spatio-Temporal Transformer for Visual Tracking](https://mlsanity.com/paper/2103.17154/) (2021)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Learning Target Candidate Association to Keep Track of What Not to Track](https://mlsanity.com/paper/2103.16556/) (2021)
- [Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark](https://mlsanity.com/paper/2103.16746/) (2021)
- [Transforming Model Prediction for Tracking](https://mlsanity.com/paper/2203.11192/) (2022)
- [STMTrack: Template-free Visual Tracking with Space-time Memory Networks](https://mlsanity.com/paper/2104.00324/) (2021)
- [Transformer Tracking with Cyclic Shifting Window Attention](https://mlsanity.com/paper/2205.03806/) (2022)
- [Learn to Match: Automatic Matching Network Design for Visual Tracking](https://mlsanity.com/paper/2108.00803/) (2021)
- [Correlation-Aware Deep Tracking](https://mlsanity.com/paper/2203.01666/) (2022)
- [Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning](https://mlsanity.com/paper/2206.03996/) (2022)
- [Compact Transformer Tracker with Correlative Masked Modeling](https://mlsanity.com/paper/2301.10938/) (2023)
- [Elysium: Exploring Object-level Perception in Videos via MLLM](https://mlsanity.com/paper/2403.16558/) (2024)
- [Cross-modal Orthogonal High-rank Augmentation for RGB-Event Transformer-trackers](https://mlsanity.com/paper/2307.04129/) (2023)
- [Robust Visual Tracking by Segmentation](https://mlsanity.com/paper/2203.11191/) (2022)
- [Unified Transformer Tracker for Object Tracking](https://mlsanity.com/paper/2203.15175/) (2022)
- [Saliency-Associated Object Tracking](https://mlsanity.com/paper/2108.03637/) (2021)
- [On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2102.10454/) (2021)
- [Self-Supervised Video Representation Learning with Meta-Contrastive Network](https://mlsanity.com/paper/2108.08426/) (2021)
- [MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition](https://mlsanity.com/paper/2104.01876/) (2021)
- [Meta-learning approaches for few-shot learning: A survey of recent advances](https://mlsanity.com/paper/2303.07502/) (2023)
- [SPMTrack: Spatio-Temporal Parameter-Efficient Fine-Tuning with Mixture of Experts for Scalable Visual Tracking](https://mlsanity.com/paper/2503.18338/) (2025)
- [Active Learning for Deep Visual Tracking](https://mlsanity.com/paper/2110.13259/) (2021)
- [Dynamic Attention guided Multi-Trajectory Analysis for Single Object Tracking](https://mlsanity.com/paper/2103.16086/) (2021)
- [Instance Tracking in 3D Scenes from Egocentric Videos](https://mlsanity.com/paper/2312.04117/) (2023)
- [Learning to Filter: Siamese Relation Network for Robust Tracking](https://mlsanity.com/paper/2104.00829/) (2021)
- [Learning Tracking Representations via Dual-Branch Fully Transformer Networks](https://mlsanity.com/paper/2112.02571/) (2021)

## Abstract (arXiv)

We consider the tracking problem as a special type of object detection problem, which we call instance detection. With proper initialization, a detector can be quickly converted into a tracker by learning the new instance from a single image. We find that model-agnostic meta-learning (MAML) offers a strategy to initialize the detector that satisfies our needs. We propose a principled three-step approach to build a high-performance tracker. First, pick any modern object detector trained with gradient descent. Second, conduct offline training (or initialization) with MAML. Third, perform domain adaptation using the initial frame. We follow this procedure to build two trackers, named Retina-MAML and FCOS-MAML, based on two modern detectors RetinaNet and FCOS. Evaluations on four benchmarks show that both trackers are competitive against state-of-the-art trackers. On OTB-100, Retina-MAML achieves the highest ever AUC of 0.712. On TrackingNet, FCOS-MAML ranks the first on the leader board with an AUC of 0.757 and the normalized precision of 0.822. Both trackers run in real-time at 40 FPS.