# Towards Class-agnostic Tracking Using Feature Decorrelation in Point Clouds

arXiv: https://arxiv.org/abs/2202.13524 · HTML: https://mlsanity.com/paper/2202.13524/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 68 references that are in the map.
- First posted on arXiv 28 Feb 2022.

## Builds on (most cited first, 30 of 37)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [SWAD: Domain Generalization by Seeking Flat Minima](https://mlsanity.com/paper/2102.08604/) (2021)
- [Deep Closest Point: Learning Representations for Point Cloud Registration](https://mlsanity.com/paper/1905.03304/) (2019)
- [PREDATOR: Registration of 3D Point Clouds with Low Overlap](https://mlsanity.com/paper/2011.13005/) (2020)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [PointNetLK: Robust & Efficient Point Cloud Registration using PointNet](https://mlsanity.com/paper/1903.05711/) (2019)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [Deep Stable Learning for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2104.07876/) (2021)
- [P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds](https://mlsanity.com/paper/2005.13888/) (2020)
- [Generalizable Person Re-identification with Relevance-aware Mixture of Experts](https://mlsanity.com/paper/2105.09156/) (2021)
- [Leveraging Shape Completion for 3D Siamese Tracking](https://mlsanity.com/paper/1903.01784/) (2019)
- [PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds](https://mlsanity.com/paper/2012.00987/) (2020)
- [Stable Learning via Sample Reweighting](https://mlsanity.com/paper/1911.12580/) (2019)
- [Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2108.04728/) (2021)
- [PointAugment: an Auto-Augmentation Framework for Point Cloud Classification](https://mlsanity.com/paper/2002.10876/) (2020)
- [PTTR: Relational 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2112.02857/) (2021)
- [Stable Prediction with Model Misspecification and Agnostic Distribution Shift](https://mlsanity.com/paper/2001.11713/) (2020)
- [Tracking by Instance Detection: A Meta-Learning Approach](https://mlsanity.com/paper/2004.00830/) (2020)
- [Siamese Instance Search for Tracking](https://mlsanity.com/paper/1605.05863/) (2016)
- [PTT: Point-Track-Transformer Module for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2108.06455/) (2021)
- [3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds](https://mlsanity.com/paper/2111.04426/) (2021)

## Built on by (most cited first, 5 of 5)

- [OST: Efficient One-stream Network for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2210.08518/) (2022)
- [iBA: Backdoor Attack on 3D Point Cloud via Reconstructing Itself](https://mlsanity.com/paper/2403.05847/) (2024)
- [Adversarial Attacks on LiDAR-Based Tracking Across Road Users: Robustness Evaluation and Target-Aware Black-Box Method](https://mlsanity.com/paper/2410.20893/) (2024)
- [Robust Single Object Tracking in LiDAR Point Clouds under Adverse Weather Conditions](https://mlsanity.com/paper/2501.07133/) (2025)
- [MambaTrack3D: A State Space Model Framework for LiDAR-Based Object Tracking under High Temporal Variation](https://mlsanity.com/paper/2511.15077/) (2025)

## Abstract (arXiv)

Single object tracking in point clouds has been attracting more and more attention owing to the presence of LiDAR sensors in 3D vision. However, the existing methods based on deep neural networks focus mainly on training different models for different categories, which makes them unable to perform well in real-world applications when encountering classes unseen during the training phase. In this work, we investigate a more challenging task in the LiDAR point clouds, class-agnostic tracking, where a general model is supposed to be learned for any specified targets of both observed and unseen categories. In particular, we first investigate the class-agnostic performances of the state-of-the-art trackers via exposing the unseen categories to them during testing, finding that a key factor for class-agnostic tracking is how to constrain fused features between the template and search region to maintain generalization when the distribution is shifted from observed to unseen classes. Therefore, we propose a feature decorrelation method to address this problem, which eliminates the spurious correlations of the fused features through a set of learned weights and further makes the search region consistent among foreground points and distinctive between foreground and background points. Experiments on the KITTI and NuScenes demonstrate that the proposed method can achieve considerable improvements by benchmarking against the advanced trackers P2B and BAT, especially when tracking unseen objects.