# Immortal Tracker: Tracklet Never Dies

arXiv: https://arxiv.org/abs/2111.13672 · HTML: https://mlsanity.com/paper/2111.13672/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 21 of 21)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [RangeDet:In Defense of Range View for LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2103.10039/) (2021)
- [LiDAR R-CNN: An Efficient and Universal 3D Object Detector](https://mlsanity.com/paper/2103.15297/) (2021)
- [GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learning](https://mlsanity.com/paper/2006.07327/) (2020)
- [SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking](https://mlsanity.com/paper/2111.09621/) (2021)
- [The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes](https://mlsanity.com/paper/1903.01568/) (2019)
- [Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Driving](https://mlsanity.com/paper/2012.13755/) (2020)
- [Probabilistic 3D Multi-Object Tracking for Autonomous Driving](https://mlsanity.com/paper/2001.05673/) (2020)
- [3D Multi-Object Tracking: A Baseline and New Evaluation Metrics](https://mlsanity.com/paper/1907.03961/) (2019)
- [Learnable Graph Matching: Incorporating Graph Partitioning with Deep Feature Learning for Multiple Object Tracking](https://mlsanity.com/paper/2103.16178/) (2021)
- [Improving 3D Object Detection through Progressive Population Based Augmentation](https://mlsanity.com/paper/2004.00831/) (2020)
- [Learnable Online Graph Representations for 3D Multi-Object Tracking](https://mlsanity.com/paper/2104.11747/) (2021)
- [Score refinement for confidence-based 3D multi-object tracking](https://mlsanity.com/paper/2107.04327/) (2021)

## Built on by (most cited first, 30 of 42)

- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Enhancing End-to-End Autonomous Driving with Latent World Model](https://mlsanity.com/paper/2406.08481/) (2024)
- [NeuRAD: Neural Rendering for Autonomous Driving](https://mlsanity.com/paper/2311.15260/) (2023)
- [Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking](https://mlsanity.com/paper/2302.03802/) (2023)
- [ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association](https://mlsanity.com/paper/2405.08909/) (2024)
- [DetZero: Rethinking Offboard 3D Object Detection with Long-term Sequential Point Clouds](https://mlsanity.com/paper/2306.06023/) (2023)
- [Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detection](https://mlsanity.com/paper/2304.12315/) (2023)
- [3DMOTFormer: Graph Transformer for Online 3D Multi-Object Tracking](https://mlsanity.com/paper/2308.06635/) (2023)
- [ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2303.15334/) (2023)
- [MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusion](https://mlsanity.com/paper/2306.17000/) (2023)
- [4D Unsupervised Object Discovery](https://mlsanity.com/paper/2210.04801/) (2022)
- [DualAD: Disentangling the Dynamic and Static World for End-to-End Driving](https://mlsanity.com/paper/2406.06264/) (2024)
- [MCTrack: A Unified 3D Multi-Object Tracking Framework for Autonomous Driving](https://mlsanity.com/paper/2409.16149/) (2024)
- [GNN-PMB: A Simple but Effective Online 3D Multi-Object Tracker without Bells and Whistles](https://mlsanity.com/paper/2206.10255/) (2022)
- [TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses](https://mlsanity.com/paper/2306.05888/) (2023)
- [Neural Enhanced Belief Propagation for Multiobject Tracking](https://mlsanity.com/paper/2212.08340/) (2022)
- [Minkowski Tracker: A Sparse Spatio-Temporal R-CNN for Joint Object Detection and Tracking](https://mlsanity.com/paper/2208.10056/) (2022)
- [LEGO: Learning and Graph-Optimized Modular Tracker for Online Multi-Object Tracking with Point Clouds](https://mlsanity.com/paper/2308.09908/) (2023)
- [RobMOT: Robust 3D Multi-Object Tracking by Observational Noise and State Estimation Drift Mitigation on LiDAR PointCloud](https://mlsanity.com/paper/2405.11536/) (2024)
- [3D Video Object Detection with Learnable Object-Centric Global Optimization](https://mlsanity.com/paper/2303.15416/) (2023)
- [Offline Tracking with Object Permanence](https://mlsanity.com/paper/2310.01288/) (2023)
- [ZOPP: A Framework of Zero-shot Offboard Panoptic Perception for Autonomous Driving](https://mlsanity.com/paper/2411.05311/) (2024)
- [TruckDrive: Long-Range Autonomous Highway Driving Dataset](https://mlsanity.com/paper/2603.02413/) (2026)
- [Streaming Motion Forecasting for Autonomous Driving](https://mlsanity.com/paper/2310.01351/) (2023)
- [3D Single-object Tracking in Point Clouds with High Temporal Variation](https://mlsanity.com/paper/2408.02049/) (2024)
- [3D Multi-Object Tracking Based on Uncertainty-Guided Data Association](https://mlsanity.com/paper/2303.01786/) (2023)
- [Frame Fusion with Vehicle Motion Prediction for 3D Object Detection](https://mlsanity.com/paper/2306.10699/) (2023)
- [Towards Flexible 3D Perception: Object-Centric Occupancy Completion Augments 3D Object Detection](https://mlsanity.com/paper/2412.05154/) (2024)
- [Bayesian Multiobject Tracking With Neural-Enhanced Motion and Measurement Models](https://mlsanity.com/paper/2506.18124/) (2025)
- [Tracking Objects with 3D Representation from Videos](https://mlsanity.com/paper/2306.05416/) (2023)

## Abstract (arXiv)

Previous online 3D Multi-Object Tracking(3DMOT) methods terminate a tracklet when it is not associated with new detections for a few frames. But if an object just goes dark, like being temporarily occluded by other objects or simply getting out of FOV, terminating a tracklet prematurely will result in an identity switch. We reveal that premature tracklet termination is the main cause of identity switches in modern 3DMOT systems. To address this, we propose Immortal Tracker, a simple tracking system that utilizes trajectory prediction to maintain tracklets for objects gone dark. We employ a simple Kalman filter for trajectory prediction and preserve the tracklet by prediction when the target is not visible. With this method, we can avoid 96% vehicle identity switches resulting from premature tracklet termination. Without any learned parameters, our method achieves a mismatch ratio at the 0.0001 level and competitive MOTA for the vehicle class on the Waymo Open Dataset test set. Our mismatch ratio is tens of times lower than any previously published method. Similar results are reported on nuScenes. We believe the proposed Immortal Tracker can offer a simple yet powerful solution for pushing the limit of 3DMOT. Our code is available at https://github.com/ImmortalTracker/ImmortalTracker.