# Factor Graph based 3D Multi-Object Tracking in Point Clouds

arXiv: https://arxiv.org/abs/2008.05309 · HTML: https://mlsanity.com/paper/2008.05309/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2020.

## Built on by (most cited first, 9 of 9)

- [SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking](https://mlsanity.com/paper/2111.09621/) (2021)
- [4D Panoptic LiDAR Segmentation](https://mlsanity.com/paper/2102.12472/) (2021)
- [3D Multi-Object Tracking Using Graph Neural Networks with Cross-Edge Modality Attention](https://mlsanity.com/paper/2203.10926/) (2022)
- [TrackMPNN: A Message Passing Graph Neural Architecture for Multi-Object Tracking](https://mlsanity.com/paper/2101.04206/) (2021)
- [RaTrack: Moving Object Detection and Tracking with 4D Radar Point Cloud](https://mlsanity.com/paper/2309.09737/) (2023)
- [MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation](https://mlsanity.com/paper/2110.09481/) (2021)
- [TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses](https://mlsanity.com/paper/2306.05888/) (2023)
- [Minkowski Tracker: A Sparse Spatio-Temporal R-CNN for Joint Object Detection and Tracking](https://mlsanity.com/paper/2208.10056/) (2022)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)

## Abstract (arXiv)

Accurate and reliable tracking of multiple moving objects in 3D space is an essential component of urban scene understanding. This is a challenging task because it requires the assignment of detections in the current frame to the predicted objects from the previous one. Existing filter-based approaches tend to struggle if this initial assignment is not correct, which can happen easily. We propose a novel optimization-based approach that does not rely on explicit and fixed assignments. Instead, we represent the result of an off-the-shelf 3D object detector as Gaussian mixture model, which is incorporated in a factor graph framework. This gives us the flexibility to assign all detections to all objects simultaneously. As a result, the assignment problem is solved implicitly and jointly with the 3D spatial multi-object state estimation using non-linear least squares optimization. Despite its simplicity, the proposed algorithm achieves robust and reliable tracking results and can be applied for offline as well as online tracking. We demonstrate its performance on the real world KITTI tracking dataset and achieve better results than many state-of-the-art algorithms. Especially the consistency of the estimated tracks is superior offline as well as online.