# Video Annotation for Visual Tracking via Selection and Refinement

arXiv: https://arxiv.org/abs/2108.03821 · HTML: https://mlsanity.com/paper/2108.03821/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 9 Aug 2021.

## Builds on (most cited first, 24 of 24)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [Fast Online Object Tracking and Segmentation: A Unifying Approach](https://mlsanity.com/paper/1812.05050/) (2018)
- [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020)
- [Ocean: Object-aware Anchor-free Tracking](https://mlsanity.com/paper/2006.10721/) (2020)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [Siamese Box Adaptive Network for Visual Tracking](https://mlsanity.com/paper/2003.06761/) (2020)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines](https://mlsanity.com/paper/1911.06188/) (2019)
- [BoxInst: High-Performance Instance Segmentation with Box Annotations](https://mlsanity.com/paper/2012.02310/) (2020)
- [SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking](https://mlsanity.com/paper/1911.07241/) (2019)
- [YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video](https://mlsanity.com/paper/1702.00824/) (2017)
- [Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box Estimation](https://mlsanity.com/paper/2007.02024/) (2020)
- [Long-term Tracking in the Wild: A Benchmark](https://mlsanity.com/paper/1803.09502/) (2018)
- [PathTrack: Fast Trajectory Annotation with Path Supervision](https://mlsanity.com/paper/1703.02437/) (2017)
- [ScribbleBox: Interactive Annotation Framework for Video Object Segmentation](https://mlsanity.com/paper/2008.09721/) (2020)
- [Efficient video annotation with visual interpolation and frame selection guidance](https://mlsanity.com/paper/2012.12554/) (2020)
- [Super-Trajectory for Video Segmentation](https://mlsanity.com/paper/1702.08634/) (2017)

## Built on by (most cited first, 5 of 5)

- [WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking](https://mlsanity.com/paper/2201.07425/) (2022)
- [Learning the What and How of Annotation in Video Object Segmentation](https://mlsanity.com/paper/2311.04414/) (2023)
- [Objects do not disappear: Video object detection by single-frame object location anticipation](https://mlsanity.com/paper/2308.04770/) (2023)
- [SPAMming Labels: Efficient Annotations for the Trackers of Tomorrow](https://mlsanity.com/paper/2404.11426/) (2024)
- [Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision](https://mlsanity.com/paper/2603.18298/) (2026)

## Abstract (arXiv)

Deep learning based visual trackers entail offline pre-training on large volumes of video datasets with accurate bounding box annotations that are labor-expensive to achieve. We present a new framework to facilitate bounding box annotations for video sequences, which investigates a selection-and-refinement strategy to automatically improve the preliminary annotations generated by tracking algorithms. A temporal assessment network (T-Assess Net) is proposed which is able to capture the temporal coherence of target locations and select reliable tracking results by measuring their quality. Meanwhile, a visual-geometry refinement network (VG-Refine Net) is also designed to further enhance the selected tracking results by considering both target appearance and temporal geometry constraints, allowing inaccurate tracking results to be corrected. The combination of the above two networks provides a principled approach to ensure the quality of automatic video annotation. Experiments on large scale tracking benchmarks demonstrate that our method can deliver highly accurate bounding box annotations and significantly reduce human labor by 94.0%, yielding an effective means to further boost tracking performance with augmented training data.