# Point Cloud Registration-Driven Robust Feature Matching for 3D Siamese Object Tracking arXiv: https://arxiv.org/abs/2209.06395 ยท HTML: https://mlsanity.com/paper/2209.06395/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Builds on 31 of its 61 references that are in the map. - First posted on arXiv 14 Sep 2022. ## Builds on (most cited first, 30 of 31) - [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019) - [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017) - [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019) - [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020) - [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018) - [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019) - [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019) - [Deep Closest Point: Learning Representations for Point Cloud Registration](https://mlsanity.com/paper/1905.03304/) (2019) - [PREDATOR: Registration of 3D Point Clouds with Low Overlap](https://mlsanity.com/paper/2011.13005/) (2020) - [PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling](https://mlsanity.com/paper/2003.00492/) (2020) - [PointNetLK: Robust & Efficient Point Cloud Registration using PointNet](https://mlsanity.com/paper/1903.05711/) (2019) - [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020) - [RPM-Net: Robust Point Matching using Learned Features](https://mlsanity.com/paper/2003.13479/) (2020) - [PRNet: Self-Supervised Learning for Partial-to-Partial Registration](https://mlsanity.com/paper/1910.12240/) (2019) - [DeepGMR: Learning Latent Gaussian Mixture Models for Registration](https://mlsanity.com/paper/2008.09088/) (2020) - [P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds](https://mlsanity.com/paper/2005.13888/) (2020) - [EagerMOT: 3D Multi-Object Tracking via Sensor Fusion](https://mlsanity.com/paper/2104.14682/) (2021) - [The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes](https://mlsanity.com/paper/1903.01568/) (2019) - [Leveraging Shape Completion for 3D Siamese Tracking](https://mlsanity.com/paper/1903.01784/) (2019) - [You Only Hypothesize Once: Point Cloud Registration with Rotation-equivariant Descriptors](https://mlsanity.com/paper/2109.00182/) (2021) - [Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2108.04728/) (2021) - [PTTR: Relational 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2112.02857/) (2021) - [PTT: Point-Track-Transformer Module for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2108.06455/) (2021) - [3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds](https://mlsanity.com/paper/2111.04426/) (2021) - [3D Multi-Object Tracking: A Baseline and New Evaluation Metrics](https://mlsanity.com/paper/1907.03961/) (2019) - [3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud](https://mlsanity.com/paper/2108.05630/) (2021) - [JRMOT: A Real-Time 3D Multi-Object Tracker and a New Large-Scale Dataset](https://mlsanity.com/paper/2002.08397/) (2020) - [Sampling Network Guided Cross-Entropy Method for Unsupervised Point Cloud Registration](https://mlsanity.com/paper/2109.06619/) (2021) - [ReAgent: Point Cloud Registration using Imitation and Reinforcement Learning](https://mlsanity.com/paper/2103.15231/) (2021) - [Mapping in a cycle: Sinkhorn regularized unsupervised learning for point cloud shapes](https://mlsanity.com/paper/2007.09594/) (2020) ## Built on by (most cited first, 2 of 2) - [Robust 3D Tracking with Quality-Aware Shape Completion](https://mlsanity.com/paper/2312.10608/) (2023) - [OST: Efficient One-stream Network for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2210.08518/) (2022) ## Abstract (arXiv) Learning robust feature matching between the template and search area is crucial for 3D Siamese tracking. The core of Siamese feature matching is how to assign high feature similarity on the corresponding points between the template and search area for precise object localization. In this paper, we propose a novel point cloud registration-driven Siamese tracking framework, with the intuition that spatially aligned corresponding points (via 3D registration) tend to achieve consistent feature representations. Specifically, our method consists of two modules, including a tracking-specific nonlocal registration module and a registration-aided Sinkhorn template-feature aggregation module. The registration module targets at the precise spatial alignment between the template and search area. The tracking-specific spatial distance constraint is proposed to refine the cross-attention weights in the nonlocal module for discriminative feature learning. Then, we use the weighted SVD to compute the rigid transformation between the template and search area, and align them to achieve the desired spatially aligned corresponding points. For the feature aggregation model, we formulate the feature matching between the transformed template and search area as an optimal transport problem and utilize the Sinkhorn optimization to search for the outlier-robust matching solution. Also, a registration-aided spatial distance map is built to improve the matching robustness in indistinguishable regions (e.g., smooth surface). Finally, guided by the obtained feature matching map, we aggregate the target information from the template into the search area to construct the target-specific feature, which is then fed into a CenterPoint-like detection head for object localization. Extensive experiments on KITTI, NuScenes and Waymo datasets verify the effectiveness of our proposed method.