# RGBD Object Tracking: An In-depth Review

arXiv: https://arxiv.org/abs/2203.14134 · HTML: https://mlsanity.com/paper/2203.14134/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 76 references that are in the map.
- First posted on arXiv 26 Mar 2022.

## Builds on (most cited first, 26 of 26)

- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Learning Spatio-Temporal Transformer for Visual Tracking](https://mlsanity.com/paper/2103.17154/) (2021)
- [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [Deeper and Wider Siamese Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1901.01660/) (2019)
- [VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows](https://mlsanity.com/paper/2108.05015/) (2021)
- [LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking](https://mlsanity.com/paper/2104.13202/) (2021)
- [Alpha-Refine: Boosting Tracking Performance by Precise Bounding Box Estimation](https://mlsanity.com/paper/2007.02024/) (2020)
- [DepthTrack : Unveiling the Power of RGBD Tracking](https://mlsanity.com/paper/2108.13962/) (2021)
- [Object Tracking by Reconstruction with View-Specific Discriminative Correlation Filters](https://mlsanity.com/paper/1811.10863/) (2018)
- [CDTB: A Color and Depth Visual Object Tracking Dataset and Benchmark](https://mlsanity.com/paper/1907.00618/) (2019)
- [DAL -- A Deep Depth-aware Long-term Tracker](https://mlsanity.com/paper/1912.00660/) (2019)
- [RGBD Datasets: Past, Present and Future](https://mlsanity.com/paper/1604.00999/) (2016)
- [Multi-modal Visual Tracking: Review and Experimental Comparison](https://mlsanity.com/paper/2012.04176/) (2020)
- [TSDM: Tracking by SiamRPN++ with a Depth-refiner and a Mask-generator](https://mlsanity.com/paper/2005.04063/) (2020)
- [Depth Masked Discriminative Correlation Filter](https://mlsanity.com/paper/1802.09227/) (2018)
- [A CRF-based Framework for Tracklet Inactivation in Online Multi-Object Tracking](https://mlsanity.com/paper/2011.14594/) (2020)
- [Multiple Human Tracking in RGB-D Data: A Survey](https://mlsanity.com/paper/1606.04450/) (2016)

## Built on by (most cited first, 11 of 11)

- [Single-Model and Any-Modality for Video Object Tracking](https://mlsanity.com/paper/2311.15851/) (2023)
- [XTrack: Multimodal Training Boosts RGB-X Video Object Trackers](https://mlsanity.com/paper/2405.17773/) (2024)
- [Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography](https://mlsanity.com/paper/2212.12324/) (2022)
- [Awesome Multi-modal Object Tracking](https://mlsanity.com/paper/2405.14200/) (2024)
- [Learning Dual-Fused Modality-Aware Representations for RGBD Tracking](https://mlsanity.com/paper/2211.03055/) (2022)
- [Video Understanding: From Geometry and Semantics to Unified Models](https://mlsanity.com/paper/2603.17840/) (2026)
- [Visual Object Tracking across Diverse Data Modalities: A Review](https://mlsanity.com/paper/2412.09991/) (2024)
- [Omni Survey for Multimodality Analysis in Visual Object Tracking](https://mlsanity.com/paper/2508.13000/) (2025)
- [Enhanced Object Tracking by Self-Supervised Auxiliary Depth Estimation Learning](https://mlsanity.com/paper/2405.14195/) (2024)
- [Depth Attention for Robust RGB Tracking](https://mlsanity.com/paper/2410.20395/) (2024)
- [MUOT_3M: A 3 Million Frame Multimodal Underwater Benchmark and the MUTrack Tracking Method](https://mlsanity.com/paper/2602.18006/) (2026)

## Abstract (arXiv)

RGBD object tracking is gaining momentum in computer vision research thanks to the development of depth sensors. Although numerous RGBD trackers have been proposed with promising performance, an in-depth review for comprehensive understanding of this area is lacking. In this paper, we firstly review RGBD object trackers from different perspectives, including RGBD fusion, depth usage, and tracking framework. Then, we summarize the existing datasets and the evaluation metrics. We benchmark a representative set of RGBD trackers, and give detailed analyses based on their performances. Particularly, we are the first to provide depth quality evaluation and analysis of tracking results in depth-friendly scenarios in RGBD tracking. For long-term settings in most RGBD tracking videos, we give an analysis of trackers' performance on handling target disappearance. To enable better understanding of RGBD trackers, we propose robustness evaluation against input perturbations. Finally, we summarize the challenges and provide open directions for this community. All resources are publicly available at https://github.com/memoryunreal/RGBD-tracking-review.