# 3D Siamese Transformer Network for Single Object Tracking on Point Clouds

arXiv: https://arxiv.org/abs/2207.11995 · HTML: https://mlsanity.com/paper/2207.11995/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 81 references that are in the map.
- First posted on arXiv 25 Jul 2022.

## Builds on (most cited first, 30 of 53)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [3D Object Detection with Pointformer](https://mlsanity.com/paper/2012.11409/) (2020)
- [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018)
- [End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures](https://mlsanity.com/paper/1911.08460/) (2019)

## Built on by (most cited first, 30 of 44)

- [CXTrack: Improving 3D Point Cloud Tracking with Contextual Information](https://mlsanity.com/paper/2211.08542/) (2022)
- [MBPTrack: Improving 3D Point Cloud Tracking with Memory Networks and Box Priors](https://mlsanity.com/paper/2303.05071/) (2023)
- [DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Tasks](https://mlsanity.com/paper/2304.00571/) (2023)
- [OSP2B: One-Stage Point-to-Box Network for 3D Siamese Tracking](https://mlsanity.com/paper/2304.11584/) (2023)
- [Synchronize Feature Extracting and Matching: A Single Branch Framework for 3D Object Tracking](https://mlsanity.com/paper/2308.12549/) (2023)
- [Towards Category Unification of 3D Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2401.11204/) (2024)
- [Correlation Pyramid Network for 3D Single Object Tracking](https://mlsanity.com/paper/2305.09195/) (2023)
- [STTracker: Spatio-Temporal Tracker for 3D Single Object Tracking](https://mlsanity.com/paper/2306.17440/) (2023)
- [M3SOT: Multi-frame, Multi-field, Multi-space 3D Single Object Tracking](https://mlsanity.com/paper/2312.06117/) (2023)
- [P2P: Part-to-Part Motion Cues Guide a Strong Tracking Framework for LiDAR Point Clouds](https://mlsanity.com/paper/2407.05238/) (2024)
- [MVCTrack: Boosting 3D Point Cloud Tracking via Multimodal-Guided Virtual Cues](https://mlsanity.com/paper/2412.02734/) (2024)
- [CompTrack: Information Bottleneck-Guided Low-Rank Dynamic Token Compression for Point Cloud Tracking](https://mlsanity.com/paper/2511.15580/) (2025)
- [Modeling Continuous Motion for 3D Point Cloud Object Tracking](https://mlsanity.com/paper/2303.07605/) (2023)
- [Robust 3D Tracking with Quality-Aware Shape Completion](https://mlsanity.com/paper/2312.10608/) (2023)
- [FocusTrack: One-Stage Focus-and-Suppress Framework for 3D Point Cloud Object Tracking](https://mlsanity.com/paper/2602.24133/) (2026)
- [SeqTrack3D: Exploring Sequence Information for Robust 3D Point Cloud Tracking](https://mlsanity.com/paper/2402.16249/) (2024)
- [OST: Efficient One-stream Network for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2210.08518/) (2022)
- [VoxelTrack: Exploring Voxel Representation for 3D Point Cloud Object Tracking](https://mlsanity.com/paper/2408.02263/) (2024)
- [GLT-T++: Global-Local Transformer for 3D Siamese Tracking with Ranking Loss](https://mlsanity.com/paper/2304.00242/) (2023)
- [3D Single-object Tracking in Point Clouds with High Temporal Variation](https://mlsanity.com/paper/2408.02049/) (2024)
- [TrackAny3D: Transferring Pretrained 3D Models for Category-unified 3D Point Cloud Tracking](https://mlsanity.com/paper/2507.19908/) (2025)
- [Beyond Frame-wise Tracking: A Trajectory-based Paradigm for Efficient Point Cloud Tracking](https://mlsanity.com/paper/2509.11453/) (2025)
- [MixCycle: Mixup Assisted Semi-Supervised 3D Single Object Tracking with Cycle Consistency](https://mlsanity.com/paper/2303.09219/) (2023)
- [Object Re-Identification from Point Clouds](https://mlsanity.com/paper/2305.10210/) (2023)
- [Motion-to-Matching: A Mixed Paradigm for 3D Single Object Tracking](https://mlsanity.com/paper/2308.11875/) (2023)
- [BEVTrack: A Simple and Strong Baseline for 3D Single Object Tracking in Bird's-Eye View](https://mlsanity.com/paper/2309.02185/) (2023)
- [FlowTrack: Point-level Flow Network for 3D Single Object Tracking](https://mlsanity.com/paper/2407.01959/) (2024)
- [PillarTrack:Boosting Pillar Representation for Transformer-based 3D Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2404.07495/) (2024)
- [GSOT3D: Towards Generic 3D Single Object Tracking in the Wild](https://mlsanity.com/paper/2412.02129/) (2024)
- [Visual Object Tracking across Diverse Data Modalities: A Review](https://mlsanity.com/paper/2412.09991/) (2024)

## Abstract (arXiv)

Siamese network based trackers formulate 3D single object tracking as cross-correlation learning between point features of a template and a search area. Due to the large appearance variation between the template and search area during tracking, how to learn the robust cross correlation between them for identifying the potential target in the search area is still a challenging problem. In this paper, we explicitly use Transformer to form a 3D Siamese Transformer network for learning robust cross correlation between the template and the search area of point clouds. Specifically, we develop a Siamese point Transformer network to learn shape context information of the target. Its encoder uses self-attention to capture non-local information of point clouds to characterize the shape information of the object, and the decoder utilizes cross-attention to upsample discriminative point features. After that, we develop an iterative coarse-to-fine correlation network to learn the robust cross correlation between the template and the search area. It formulates the cross-feature augmentation to associate the template with the potential target in the search area via cross attention. To further enhance the potential target, it employs the ego-feature augmentation that applies self-attention to the local k-NN graph of the feature space to aggregate target features. Experiments on the KITTI, nuScenes, and Waymo datasets show that our method achieves state-of-the-art performance on the 3D single object tracking task.