# VITAL: VIsual Tracking via Adversarial Learning

arXiv: https://arxiv.org/abs/1804.04273 · HTML: https://mlsanity.com/paper/1804.04273/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2018.

## Built on by (most cited first, 30 of 65)

- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examples](https://mlsanity.com/paper/2103.05905/) (2021)
- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark](https://mlsanity.com/paper/2103.16746/) (2021)
- [STMTrack: Template-free Visual Tracking with Space-time Memory Networks](https://mlsanity.com/paper/2104.00324/) (2021)
- [Robust Object Modeling for Visual Tracking](https://mlsanity.com/paper/2308.05140/) (2023)
- [Learn to Match: Automatic Matching Network Design for Visual Tracking](https://mlsanity.com/paper/2108.00803/) (2021)
- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning](https://mlsanity.com/paper/2403.09634/) (2024)
- [Disentangled Cycle Consistency for Highly-realistic Virtual Try-On](https://mlsanity.com/paper/2103.09479/) (2021)
- [Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric](https://mlsanity.com/paper/2211.11010/) (2022)
- [SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Tracking](https://mlsanity.com/paper/2403.16002/) (2024)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [Generative-based Fusion Mechanism for Multi-Modal Tracking](https://mlsanity.com/paper/2309.01728/) (2023)
- [Unified Transformer Tracker for Object Tracking](https://mlsanity.com/paper/2203.15175/) (2022)
- [DeFLOCNet: Deep Image Editing via Flexible Low-level Controls](https://mlsanity.com/paper/2103.12723/) (2021)
- [WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking](https://mlsanity.com/paper/2201.07425/) (2022)
- [IoU Attack: Towards Temporally Coherent Black-Box Adversarial Attack for Visual Object Tracking](https://mlsanity.com/paper/2103.14938/) (2021)
- [Visual Object Tracking in First Person Vision](https://mlsanity.com/paper/2209.13502/) (2022)
- [Learning to Track Objects from Unlabeled Videos](https://mlsanity.com/paper/2108.12711/) (2021)
- [Is First Person Vision Challenging for Object Tracking?](https://mlsanity.com/paper/2108.13665/) (2021)
- [Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking](https://mlsanity.com/paper/2304.14394/) (2023)
- [Learning to Filter: Siamese Relation Network for Robust Tracking](https://mlsanity.com/paper/2104.00829/) (2021)
- [WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmark](https://mlsanity.com/paper/2405.19818/) (2024)
- [What You Have is What You Track: Adaptive and Robust Multimodal Tracking](https://mlsanity.com/paper/2507.05899/) (2025)
- [Tiny Object Tracking: A Large-scale Dataset and A Baseline](https://mlsanity.com/paper/2202.05659/) (2022)
- [Mobile Augmented Reality: User Interfaces, Frameworks, and Intelligence](https://mlsanity.com/paper/2106.08710/) (2021)
- [CSTrack: Enhancing RGB-X Tracking via Compact Spatiotemporal Features](https://mlsanity.com/paper/2505.19434/) (2025)
- [ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking](https://mlsanity.com/paper/2508.05221/) (2025)
- [Cross-Modal Object Tracking via Modality-Aware Fusion Network and A Large-Scale Dataset](https://mlsanity.com/paper/2312.14446/) (2023)

## Abstract (arXiv)

The tracking-by-detection framework consists of two stages, i.e., drawing samples around the target object in the first stage and classifying each sample as the target object or as background in the second stage. The performance of existing trackers using deep classification networks is limited by two aspects. First, the positive samples in each frame are highly spatially overlapped, and they fail to capture rich appearance variations. Second, there exists extreme class imbalance between positive and negative samples. This paper presents the VITAL algorithm to address these two problems via adversarial learning. To augment positive samples, we use a generative network to randomly generate masks, which are applied to adaptively dropout input features to capture a variety of appearance changes. With the use of adversarial learning, our network identifies the mask that maintains the most robust features of the target objects over a long temporal span. In addition, to handle the issue of class imbalance, we propose a high-order cost sensitive loss to decrease the effect of easy negative samples to facilitate training the classification network. Extensive experiments on benchmark datasets demonstrate that the proposed tracker performs favorably against state-of-the-art approaches.