# Accelerated Video Annotation driven by Deep Detector and Tracker

arXiv: https://arxiv.org/abs/2302.09590 · HTML: https://mlsanity.com/paper/2302.09590/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 14 references that are in the map.
- First posted on arXiv 19 Feb 2023.

## Builds on (most cited first, 6 of 6)

- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Recent Advances in Deep Learning for Object Detection](https://mlsanity.com/paper/1908.03673/) (2019)
- [Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook](https://mlsanity.com/paper/2112.02838/) (2021)
- [Efficient video annotation with visual interpolation and frame selection guidance](https://mlsanity.com/paper/2012.12554/) (2020)
- [Deep Neural Network-based Cooperative Visual Tracking through Multiple Micro Aerial Vehicles](https://mlsanity.com/paper/1802.01346/) (2018)

## Built on by (most cited first, 1 of 1)

- [Synthetic Data-based Detection of Zebras in Drone Imagery](https://mlsanity.com/paper/2305.00432/) (2023)

## Abstract (arXiv)

Annotating object ground truth in videos is vital for several downstream tasks in robot perception and machine learning, such as for evaluating the performance of an object tracker or training an image-based object detector. The accuracy of the annotated instances of the moving objects on every image frame in a video is crucially important. Achieving that through manual annotations is not only very time consuming and labor intensive, but is also prone to high error rate. State-of-the-art annotation methods depend on manually initializing the object bounding boxes only in the first frame and then use classical tracking methods, e.g., adaboost, or kernelized correlation filters, to keep track of those bounding boxes. These can quickly drift, thereby requiring tedious manual supervision. In this paper, we propose a new annotation method which leverages a combination of a learning-based detector (SSD) and a learning-based tracker (RE3). Through this, we significantly reduce annotation drifts, and, consequently, the required manual supervision. We validate our approach through annotation experiments using our proposed annotation method and existing baselines on a set of drone video frames. Source code and detailed information on how to run the annotation program can be found at https://github.com/robot-perception-group/smarter-labelme