# Model-free Vehicle Tracking and State Estimation in Point Cloud Sequences

arXiv: https://arxiv.org/abs/2103.06028 · HTML: https://mlsanity.com/paper/2103.06028/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 10 Mar 2021.

## Builds on (most cited first, 19 of 19)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [PIXOR: Real-time 3D Object Detection from Point Clouds](https://mlsanity.com/paper/1902.06326/) (2019)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World](https://mlsanity.com/paper/2006.09348/) (2020)
- [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015)
- [P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds](https://mlsanity.com/paper/2005.13888/) (2020)
- [Leveraging Shape Completion for 3D Siamese Tracking](https://mlsanity.com/paper/1903.01784/) (2019)
- [Weakly-supervised 3D Shape Completion in the Wild](https://mlsanity.com/paper/2008.09110/) (2020)
- [PointRGCN: Graph Convolution Networks for 3D Vehicles Detection Refinement](https://mlsanity.com/paper/1911.12236/) (2019)
- [AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objects](https://mlsanity.com/paper/1910.04668/) (2019)
- [F-Siamese Tracker: A Frustum-based Double Siamese Network for 3D Single Object Tracking](https://mlsanity.com/paper/2010.11510/) (2020)
- [Joint Pose and Shape Estimation of Vehicles from LiDAR Data](https://mlsanity.com/paper/2009.03964/) (2020)

## Built on by (most cited first, 25 of 25)

- [SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking](https://mlsanity.com/paper/2111.09621/) (2021)
- [Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving](https://mlsanity.com/paper/2210.08061/) (2022)
- [3D Siamese Transformer Network for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2207.11995/) (2022)
- [CXTrack: Improving 3D Point Cloud Tracking with Contextual Information](https://mlsanity.com/paper/2211.08542/) (2022)
- [Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detection](https://mlsanity.com/paper/2304.12315/) (2023)
- [MBPTrack: Improving 3D Point Cloud Tracking with Memory Networks and Box Priors](https://mlsanity.com/paper/2303.05071/) (2023)
- [MV-Map: Offboard HD-Map Generation with Multi-view Consistency](https://mlsanity.com/paper/2305.08851/) (2023)
- [Offboard Occupancy Refinement with Hybrid Propagation for Autonomous Driving](https://mlsanity.com/paper/2403.08504/) (2024)
- [SpOT: Spatiotemporal Modeling for 3D Object Tracking](https://mlsanity.com/paper/2207.05856/) (2022)
- [M3SOT: Multi-frame, Multi-field, Multi-space 3D Single Object Tracking](https://mlsanity.com/paper/2312.06117/) (2023)
- [CompTrack: Information Bottleneck-Guided Low-Rank Dynamic Token Compression for Point Cloud Tracking](https://mlsanity.com/paper/2511.15580/) (2025)
- [LabelFormer: Object Trajectory Refinement for Offboard Perception from LiDAR Point Clouds](https://mlsanity.com/paper/2311.01444/) (2023)
- [FocusTrack: One-Stage Focus-and-Suppress Framework for 3D Point Cloud Object Tracking](https://mlsanity.com/paper/2602.24133/) (2026)
- [Online Adaptation for Implicit Object Tracking and Shape Reconstruction in the Wild](https://mlsanity.com/paper/2111.12728/) (2021)
- [Sequential Point Clouds: A Survey](https://mlsanity.com/paper/2204.09337/) (2022)
- [Temporal Point Cloud Completion with Pose Disturbance](https://mlsanity.com/paper/2202.03084/) (2022)
- [Towards Class-agnostic Tracking Using Feature Decorrelation in Point Clouds](https://mlsanity.com/paper/2202.13524/) (2022)
- [3D Single-object Tracking in Point Clouds with High Temporal Variation](https://mlsanity.com/paper/2408.02049/) (2024)
- [BEVTrack: A Simple and Strong Baseline for 3D Single Object Tracking in Bird's-Eye View](https://mlsanity.com/paper/2309.02185/) (2023)
- [Multi-Correlation Siamese Transformer Network with Dense Connection for 3D Single Object Tracking](https://mlsanity.com/paper/2312.11051/) (2023)
- [SiamMo: Siamese Motion-Centric 3D Object Tracking](https://mlsanity.com/paper/2408.01688/) (2024)
- [Adversarial Attacks on LiDAR-Based Tracking Across Road Users: Robustness Evaluation and Target-Aware Black-Box Method](https://mlsanity.com/paper/2410.20893/) (2024)
- [Are All Point Clouds Suitable for Completion? Weakly Supervised Quality Evaluation Network for Point Cloud Completion](https://mlsanity.com/paper/2303.01804/) (2023)
- [Spatio-Temporal Bi-directional Cross-frame Memory for Distractor Filtering Point Cloud Single Object Tracking](https://mlsanity.com/paper/2403.15831/) (2024)
- [EasyTrack: Efficient and Compact One-stream 3D Point Clouds Tracker](https://mlsanity.com/paper/2404.05960/) (2024)

## Abstract (arXiv)

Estimating the states of surrounding traffic participants stays at the core of autonomous driving. In this paper, we study a novel setting of this problem: model-free single-object tracking (SOT), which takes the object state in the first frame as input, and jointly solves state estimation and tracking in subsequent frames. The main purpose for this new setting is to break the strong limitation of the popular "detection and tracking" scheme in multi-object tracking. Moreover, we notice that shape completion by overlaying the point clouds, which is a by-product of our proposed task, not only improves the performance of state estimation but also has numerous applications. As no benchmark for this task is available so far, we construct a new dataset LiDAR-SOT and corresponding evaluation protocols based on the Waymo Open dataset. We then propose an optimization-based algorithm called SOTracker involving point cloud registration, vehicle shapes, correspondence, and motion priors. Our quantitative and qualitative results prove the effectiveness of our SOTracker and reveal the challenging cases for SOT in point clouds, including the sparsity of LiDAR data, abrupt motion variation, etc. Finally, we also explore how the proposed task and algorithm may benefit other autonomous driving applications, including simulating LiDAR scans, generating motion data, and annotating optical flow. The code and protocols for our benchmark and algorithm are available at https://github.com/TuSimple/LiDAR_SOT/. A video demonstration is at https://www.youtube.com/watch?v=BpHixKs91i8.