# Synchronize Feature Extracting and Matching: A Single Branch Framework for 3D Object Tracking

arXiv: https://arxiv.org/abs/2308.12549 · HTML: https://mlsanity.com/paper/2308.12549/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 55 references that are in the map.
- First posted on arXiv 24 Aug 2023.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Learning Spatio-Temporal Transformer for Visual Tracking](https://mlsanity.com/paper/2103.17154/) (2021)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [Learning Multi-Domain Convolutional Neural Networks for Visual Tracking](https://mlsanity.com/paper/1510.07945/) (2015)
- [SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines](https://mlsanity.com/paper/1911.06188/) (2019)
- [Deeper and Wider Siamese Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1901.01660/) (2019)
- [SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking](https://mlsanity.com/paper/1911.07241/) (2019)
- [Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking](https://mlsanity.com/paper/2203.05328/) (2022)
- [Peeking into the Future: Predicting Future Person Activities and Locations in Videos](https://mlsanity.com/paper/1902.03748/) (2019)
- [End-to-end representation learning for Correlation Filter based tracking](https://mlsanity.com/paper/1704.06036/) (2017)
- [Learning to Track at 100 FPS with Deep Regression Networks](https://mlsanity.com/paper/1604.01802/) (2016)
- [Interpolated Convolutional Networks for 3D Point Cloud Understanding](https://mlsanity.com/paper/1908.04512/) (2019)
- [P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds](https://mlsanity.com/paper/2005.13888/) (2020)
- [Leveraging Shape Completion for 3D Siamese Tracking](https://mlsanity.com/paper/1903.01784/) (2019)
- [The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction](https://mlsanity.com/paper/1912.06445/) (2019)
- [Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2108.04728/) (2021)
- [Siamese Cascaded Region Proposal Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1812.06148/) (2018)
- [PTTR: Relational 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2112.02857/) (2021)

## Built on by (most cited first, 16 of 16)

- [Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation](https://mlsanity.com/paper/2406.09738/) (2024)
- [TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentation](https://mlsanity.com/paper/2309.07849/) (2023)
- [Towards Category Unification of 3D Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2401.11204/) (2024)
- [P2P: Part-to-Part Motion Cues Guide a Strong Tracking Framework for LiDAR Point Clouds](https://mlsanity.com/paper/2407.05238/) (2024)
- [MVCTrack: Boosting 3D Point Cloud Tracking via Multimodal-Guided Virtual Cues](https://mlsanity.com/paper/2412.02734/) (2024)
- [CompTrack: Information Bottleneck-Guided Low-Rank Dynamic Token Compression for Point Cloud Tracking](https://mlsanity.com/paper/2511.15580/) (2025)
- [FocusTrack: One-Stage Focus-and-Suppress Framework for 3D Point Cloud Object Tracking](https://mlsanity.com/paper/2602.24133/) (2026)
- [VoxelTrack: Exploring Voxel Representation for 3D Point Cloud Object Tracking](https://mlsanity.com/paper/2408.02263/) (2024)
- [TrackAny3D: Transferring Pretrained 3D Models for Category-unified 3D Point Cloud Tracking](https://mlsanity.com/paper/2507.19908/) (2025)
- [BEVTrack: A Simple and Strong Baseline for 3D Single Object Tracking in Bird's-Eye View](https://mlsanity.com/paper/2309.02185/) (2023)
- [GSOT3D: Towards Generic 3D Single Object Tracking in the Wild](https://mlsanity.com/paper/2412.02129/) (2024)
- [SiamMo: Siamese Motion-Centric 3D Object Tracking](https://mlsanity.com/paper/2408.01688/) (2024)
- [EasyTrack: Efficient and Compact One-stream 3D Point Clouds Tracker](https://mlsanity.com/paper/2404.05960/) (2024)
- [PCSTracker: Long-Term Scene Flow Estimation for Point Cloud Sequences](https://mlsanity.com/paper/2603.19762/) (2026)
- [SAVTrack: Selective Vote Aggregation for Reliability-Aware Point Cloud Tracking](https://mlsanity.com/paper/2609.16662/) (2026)
- [MAETrack: Unleashing the Potential of Pretrained Geometric Priors for 3D Single Object Tracking](https://mlsanity.com/paper/2609.16695/) (2026)

## Abstract (arXiv)

Siamese network has been a de facto benchmark framework for 3D LiDAR object tracking with a shared-parametric encoder extracting features from template and search region, respectively. This paradigm relies heavily on an additional matching network to model the cross-correlation/similarity of the template and search region. In this paper, we forsake the conventional Siamese paradigm and propose a novel single-branch framework, SyncTrack, synchronizing the feature extracting and matching to avoid forwarding encoder twice for template and search region as well as introducing extra parameters of matching network. The synchronization mechanism is based on the dynamic affinity of the Transformer, and an in-depth analysis of the relevance is provided theoretically. Moreover, based on the synchronization, we introduce a novel Attentive Points-Sampling strategy into the Transformer layers (APST), replacing the random/Farthest Points Sampling (FPS) method with sampling under the supervision of attentive relations between the template and search region. It implies connecting point-wise sampling with the feature learning, beneficial to aggregating more distinctive and geometric features for tracking with sparse points. Extensive experiments on two benchmark datasets (KITTI and NuScenes) show that SyncTrack achieves state-of-the-art performance in real-time tracking.