# GSOT3D: Towards Generic 3D Single Object Tracking in the Wild arXiv: https://arxiv.org/abs/2412.02129 ยท HTML: https://mlsanity.com/paper/2412.02129/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 41 references that are in the map. - First posted on arXiv 3 Dec 2024. ## Builds on (most cited first, 30 of 30) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019) - [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018) - [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018) - [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018) - [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018) - [Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild](https://mlsanity.com/paper/2207.10660/) (2022) - [Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations](https://mlsanity.com/paper/2012.09988/) (2020) - [Need for Speed: A Benchmark for Higher Frame Rate Object Tracking](https://mlsanity.com/paper/1703.05884/) (2017) - [LaSOT: A High-quality Large-scale Single Object Tracking Benchmark](https://mlsanity.com/paper/2009.03465/) (2020) - [Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark](https://mlsanity.com/paper/2103.16746/) (2021) - [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015) - [P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds](https://mlsanity.com/paper/2005.13888/) (2020) - [Leveraging Shape Completion for 3D Siamese Tracking](https://mlsanity.com/paper/1903.01784/) (2019) - [Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2108.04728/) (2021) - [PTTR: Relational 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2112.02857/) (2021) - [PTT: Point-Track-Transformer Module for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2108.06455/) (2021) - [Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2203.01730/) (2022) - [3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds](https://mlsanity.com/paper/2111.04426/) (2021) - [3D Siamese Transformer Network for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2207.11995/) (2022) - [CXTrack: Improving 3D Point Cloud Tracking with Contextual Information](https://mlsanity.com/paper/2211.08542/) (2022) - [VastTrack: Vast Category Visual Object Tracking](https://mlsanity.com/paper/2403.03493/) (2024) - [MBPTrack: Improving 3D Point Cloud Tracking with Memory Networks and Box Priors](https://mlsanity.com/paper/2303.05071/) (2023) - [LiDAR-Camera Calibration using 3D-3D Point correspondences](https://mlsanity.com/paper/1705.09785/) (2017) - [Synchronize Feature Extracting and Matching: A Single Branch Framework for 3D Object Tracking](https://mlsanity.com/paper/2308.12549/) (2023) - [Towards Category Unification of 3D Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2401.11204/) (2024) - [M3SOT: Multi-frame, Multi-field, Multi-space 3D Single Object Tracking](https://mlsanity.com/paper/2312.06117/) (2023) - [SeqTrack3D: Exploring Sequence Information for Robust 3D Point Cloud Tracking](https://mlsanity.com/paper/2402.16249/) (2024) - [3D Single-object Tracking in Point Clouds with High Temporal Variation](https://mlsanity.com/paper/2408.02049/) (2024) ## Built on by (most cited first, 2 of 2) - [Towards Visual Query Localization in the 3D World](https://mlsanity.com/paper/2605.01498/) (2026) - [PGL-3D: Towards Progressive Geometric Learning for 3D Visual Query Localization](https://mlsanity.com/paper/2609.33558/) (2026) ## Abstract (arXiv) In this paper, we present a novel benchmark, GSOT3D, that aims at facilitating development of generic 3D single object tracking (SOT) in the wild. Specifically, GSOT3D offers 620 sequences with 123K frames, and covers a wide selection of 54 object categories. Each sequence is offered with multiple modalities, including the point cloud (PC), RGB image, and depth. This allows GSOT3D to support various 3D tracking tasks, such as single-modal 3D SOT on PC and multi-modal 3D SOT on RGB-PC or RGB-D, and thus greatly broadens research directions for 3D object tracking. To provide highquality per-frame 3D annotations, all sequences are labeled manually with multiple rounds of meticulous inspection and refinement. To our best knowledge, GSOT3D is the largest benchmark dedicated to various generic 3D object tracking tasks. To understand how existing 3D trackers perform and to provide comparisons for future research on GSOT3D, we assess eight representative point cloud-based tracking models. Our evaluation results exhibit that these models heavily degrade on GSOT3D, and more efforts are required for robust and generic 3D object tracking. Besides, to encourage future research, we present a simple yet effective generic 3D tracker, named PROT3D, that localizes the target object via a progressive spatial-temporal network and outperforms all current solutions by a large margin. By releasing GSOT3D, we expect to advance further 3D tracking in future research and applications. Our benchmark and model as well as the evaluation results will be publicly released at our webpage https://github.com/ailovejinx/GSOT3D.