# CDTB: A Color and Depth Visual Object Tracking Dataset and Benchmark

arXiv: https://arxiv.org/abs/1907.00618 · HTML: https://mlsanity.com/paper/1907.00618/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2019.

## Built on by (most cited first, 30 of 32)

- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [DepthTrack : Unveiling the Power of RGBD Tracking](https://mlsanity.com/paper/2108.13962/) (2021)
- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [Prompting for Multi-Modal Tracking](https://mlsanity.com/paper/2207.14571/) (2022)
- [RGBD1K: A Large-scale Dataset and Benchmark for RGB-D Object Tracking](https://mlsanity.com/paper/2208.09787/) (2022)
- [MFGNet: Dynamic Modality-Aware Filter Generation for RGB-T Tracking](https://mlsanity.com/paper/2107.10433/) (2021)
- [Visual Object Tracking in First Person Vision](https://mlsanity.com/paper/2209.13502/) (2022)
- [Is First Person Vision Challenging for Object Tracking?](https://mlsanity.com/paper/2108.13665/) (2021)
- [Multi-Source Fusion and Automatic Predictor Selection for Zero-Shot Video Object Segmentation](https://mlsanity.com/paper/2108.05076/) (2021)
- [ARKitTrack: A New Diverse Dataset for Tracking Using Mobile RGB-D Data](https://mlsanity.com/paper/2303.13885/) (2023)
- [RGBD Object Tracking: An In-depth Review](https://mlsanity.com/paper/2203.14134/) (2022)
- [Tracking Small and Fast Moving Objects: A Benchmark](https://mlsanity.com/paper/2209.04284/) (2022)
- [Improving Underwater Visual Tracking With a Large Scale Dataset and Image Enhancement](https://mlsanity.com/paper/2308.15816/) (2023)
- [Adaptive Multi-source Predictor for Zero-shot Video Object Segmentation](https://mlsanity.com/paper/2303.10383/) (2023)
- [Predictive Visual Tracking: A New Benchmark and Baseline Approach](https://mlsanity.com/paper/2103.04508/) (2021)
- [Visual Object Tracking on Multi-modal RGB-D Videos: A Review](https://mlsanity.com/paper/2201.09207/) (2022)
- [Learning Dual-Fused Modality-Aware Representations for RGBD Tracking](https://mlsanity.com/paper/2211.03055/) (2022)
- [Tracking by 3D Model Estimation of Unknown Objects in Videos](https://mlsanity.com/paper/2304.06419/) (2023)
- [Depth-only Object Tracking](https://mlsanity.com/paper/2110.11679/) (2021)
- [Collaborating Vision, Depth, and Thermal Signals for Multi-Modal Tracking: Dataset and Algorithm](https://mlsanity.com/paper/2509.24741/) (2025)
- [Single Object Tracking Research: A Survey](https://mlsanity.com/paper/2204.11410/) (2022)
- [Camouflaged Object Tracking: A Benchmark](https://mlsanity.com/paper/2408.13877/) (2024)
- [Omni Survey for Multimodality Analysis in Visual Object Tracking](https://mlsanity.com/paper/2508.13000/) (2025)
- [Multiple Object Trackers in OpenCV: A Benchmark](https://mlsanity.com/paper/2110.05102/) (2021)
- [Tracking Reflected Objects: A Benchmark](https://mlsanity.com/paper/2407.05235/) (2024)
- [Heterogeneous Graph Transformer for Multiple Tiny Object Tracking in RGB-T Videos](https://mlsanity.com/paper/2412.10861/) (2024)
- [MVTD: A Benchmark Dataset for Maritime Visual Object Tracking](https://mlsanity.com/paper/2506.02866/) (2025)
- [Is Tracking really more challenging in First Person Egocentric Vision?](https://mlsanity.com/paper/2507.16015/) (2025)
- [Robot Person Following in Uniform Crowd Environment](https://mlsanity.com/paper/2205.10553/) (2022)
- [Optimizing Multi-Modality Trackers via Significance-Regularized Tuning](https://mlsanity.com/paper/2508.17488/) (2025)

## Abstract (arXiv)

A long-term visual object tracking performance evaluation methodology and a benchmark are proposed. Performance measures are designed by following a long-term tracking definition to maximize the analysis probing strength. The new measures outperform existing ones in interpretation potential and in better distinguishing between different tracking behaviors. We show that these measures generalize the short-term performance measures, thus linking the two tracking problems. Furthermore, the new measures are highly robust to temporal annotation sparsity and allow annotation of sequences hundreds of times longer than in the current datasets without increasing manual annotation labor. A new challenging dataset of carefully selected sequences with many target disappearances is proposed. A new tracking taxonomy is proposed to position trackers on the short-term/long-term spectrum. The benchmark contains an extensive evaluation of the largest number of long-term tackers and comparison to state-of-the-art short-term trackers. We analyze the influence of tracking architecture implementations to long-term performance and explore various re-detection strategies as well as influence of visual model update strategies to long-term tracking drift. The methodology is integrated in the VOT toolkit to automate experimental analysis and benchmarking and to facilitate future development of long-term trackers.