# 3D Multi-Object Tracking: A Baseline and New Evaluation Metrics

arXiv: https://arxiv.org/abs/1907.03961 · HTML: https://mlsanity.com/paper/1907.03961/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2019.

## Built on by (most cited first, 30 of 47)

- [Embracing Single Stride 3D Object Detector with Sparse Transformer](https://mlsanity.com/paper/2112.06375/) (2021)
- [Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusion](https://mlsanity.com/paper/2211.10581/) (2022)
- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [Learning to Track with Object Permanence](https://mlsanity.com/paper/2103.14258/) (2021)
- [Differentiable Raycasting for Self-supervised Occupancy Forecasting](https://mlsanity.com/paper/2210.01917/) (2022)
- [CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perception](https://mlsanity.com/paper/2304.00670/) (2023)
- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [PTTR: Relational 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2112.02857/) (2021)
- [Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving](https://mlsanity.com/paper/2210.08061/) (2022)
- [DEFT: Detection Embeddings for Tracking](https://mlsanity.com/paper/2102.02267/) (2021)
- [Immortal Tracker: Tracklet Never Dies](https://mlsanity.com/paper/2111.13672/) (2021)
- [VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild](https://mlsanity.com/paper/2108.02452/) (2021)
- [Tracking Objects as Pixel-wise Distributions](https://mlsanity.com/paper/2207.05518/) (2022)
- [STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes](https://mlsanity.com/paper/2204.01026/) (2022)
- [Unleashing HyDRa: Hybrid Fusion, Depth Consistency and Radar for Unified 3D Perception](https://mlsanity.com/paper/2403.07746/) (2024)
- [Score refinement for confidence-based 3D multi-object tracking](https://mlsanity.com/paper/2107.04327/) (2021)
- [Time3D: End-to-End Joint Monocular 3D Object Detection and Tracking for Autonomous Driving](https://mlsanity.com/paper/2205.14882/) (2022)
- [Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detection](https://mlsanity.com/paper/2304.12315/) (2023)
- [TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking](https://mlsanity.com/paper/2101.04206/) (2021)
- [ShaSTA: Modeling Shape and Spatio-Temporal Affinities for 3D Multi-Object Tracking](https://mlsanity.com/paper/2211.03919/) (2022)
- [Exploring Point-BEV Fusion for 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2208.05216/) (2022)
- [TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses](https://mlsanity.com/paper/2306.05888/) (2023)
- [MoDAR: Using Motion Forecasting for 3D Object Detection in Point Cloud Sequences](https://mlsanity.com/paper/2306.03206/) (2023)
- [Zero-Shot Open-Vocabulary Tracking with Large Pre-Trained Models](https://mlsanity.com/paper/2310.06992/) (2023)
- [SpOT: Spatiotemporal Modeling for 3D Object Tracking](https://mlsanity.com/paper/2207.05856/) (2022)
- [Depth Perspective-aware Multiple Object Tracking](https://mlsanity.com/paper/2207.04551/) (2022)
- [SpaRC: Sparse Radar-Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2411.19860/) (2024)
- [CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking](https://mlsanity.com/paper/2107.05150/) (2021)
- [Sequential Point Clouds: A Survey](https://mlsanity.com/paper/2204.09337/) (2022)
- [InterTrack: Interaction Transformer for 3D Multi-Object Tracking](https://mlsanity.com/paper/2208.08041/) (2022)

## Abstract (arXiv)

3D multi-object tracking (MOT) is an essential component for many applications such as autonomous driving and assistive robotics. Recent work on 3D MOT focuses on developing accurate systems giving less attention to practical considerations such as computational cost and system complexity. In contrast, this work proposes a simple real-time 3D MOT system. Our system first obtains 3D detections from a LiDAR point cloud. Then, a straightforward combination of a 3D Kalman filter and the Hungarian algorithm is used for state estimation and data association. Additionally, 3D MOT datasets such as KITTI evaluate MOT methods in the 2D space and standardized 3D MOT evaluation tools are missing for a fair comparison of 3D MOT methods. Therefore, we propose a new 3D MOT evaluation tool along with three new metrics to comprehensively evaluate 3D MOT methods. We show that, although our system employs a combination of classical MOT modules, we achieve state-of-the-art 3D MOT performance on two 3D MOT benchmarks (KITTI and nuScenes). Surprisingly, although our system does not use any 2D data as inputs, we achieve competitive performance on the KITTI 2D MOT leaderboard. Our proposed system runs at a rate of 207.4 FPS on the KITTI dataset, achieving the fastest speed among all modern MOT systems. To encourage standardized 3D MOT evaluation, our system and evaluation code are made publicly available at https://github.com/xinshuoweng/AB3DMOT.