# Collecting Consistently High Quality Object Tracks with Minimal Human Involvement by Using Self-Supervised Learning to Detect Tracker Errors

arXiv: https://arxiv.org/abs/2405.03643 · HTML: https://mlsanity.com/paper/2405.03643/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 6 May 2024.

## Builds on (most cited first, 18 of 18)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [Learning Spatio-Temporal Transformer for Visual Tracking](https://mlsanity.com/paper/2103.17154/) (2021)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking](https://mlsanity.com/paper/1504.01942/) (2015)
- [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015)
- [Aligning Pretraining for Detection via Object-Level Contrastive Learning](https://mlsanity.com/paper/2106.02637/) (2021)
- [Unsupervised Object-Level Representation Learning from Scene Images](https://mlsanity.com/paper/2106.11952/) (2021)
- [GMOT-40: A Benchmark for Generic Multiple Object Tracking](https://mlsanity.com/paper/2011.11858/) (2020)
- [Slow and steady feature analysis: higher order temporal coherence in video](https://mlsanity.com/paper/1506.04714/) (2015)
- [Self-Supervised Multi-Object Tracking with Cross-Input Consistency](https://mlsanity.com/paper/2111.05943/) (2021)
- [PathTrack: Fast Trajectory Annotation with Path Supervision](https://mlsanity.com/paper/1703.02437/) (2017)
- [ScribbleBox: Interactive Annotation Framework for Video Object Segmentation](https://mlsanity.com/paper/2008.09721/) (2020)
- [Efficient video annotation with visual interpolation and frame selection guidance](https://mlsanity.com/paper/2012.12554/) (2020)
- [Self-Supervised Tracking via Target-Aware Data Synthesis](https://mlsanity.com/paper/2106.10900/) (2021)
- [Learning Temporal Embeddings for Complex Video Analysis](https://mlsanity.com/paper/1505.00315/) (2015)
- [Object-Centric Representation Learning from Unlabeled Videos](https://mlsanity.com/paper/1612.00500/) (2016)

## Abstract (arXiv)

We propose a hybrid framework for consistently producing high-quality object tracks by combining an automated object tracker with little human input. The key idea is to tailor a module for each dataset to intelligently decide when an object tracker is failing and so humans should be brought in to re-localize an object for continued tracking. Our approach leverages self-supervised learning on unlabeled videos to learn a tailored representation for a target object that is then used to actively monitor its tracked region and decide when the tracker fails. Since labeled data is not needed, our approach can be applied to novel object categories. Experiments on three datasets demonstrate our method outperforms existing approaches, especially for small, fast moving, or occluded objects.