# STTracker: Spatio-Temporal Tracker for 3D Single Object Tracking

arXiv: https://arxiv.org/abs/2306.17440 · HTML: https://mlsanity.com/paper/2306.17440/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 46 references that are in the map.
- First posted on arXiv 30 Jun 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds](https://mlsanity.com/paper/1910.06528/) (2019)
- [P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds](https://mlsanity.com/paper/2005.13888/) (2020)
- [Fully Sparse 3D Object Detection](https://mlsanity.com/paper/2207.10035/) (2022)
- [Leveraging Shape Completion for 3D Siamese Tracking](https://mlsanity.com/paper/1903.01784/) (2019)
- [Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2108.04728/) (2021)
- [PTTR: Relational 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2112.02857/) (2021)
- [PTT: Point-Track-Transformer Module for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2108.06455/) (2021)
- [Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds](https://mlsanity.com/paper/2203.01730/) (2022)
- [3D Siamese Voxel-to-BEV Tracker for Sparse Point Clouds](https://mlsanity.com/paper/2111.04426/) (2021)
- [Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking](https://mlsanity.com/paper/2302.03802/) (2023)
- [3D Object Tracking with Transformer](https://mlsanity.com/paper/2110.14921/) (2021)
- [StarNet: Targeted Computation for Object Detection in Point Clouds](https://mlsanity.com/paper/1908.11069/) (2019)
- [3D Siamese Transformer Network for Single Object Tracking on Point Clouds](https://mlsanity.com/paper/2207.11995/) (2022)
- [Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks](https://mlsanity.com/paper/2110.04076/) (2021)

## Built on by (most cited first, 10 of 10)

- [MVCTrack: Boosting 3D Point Cloud Tracking via Multimodal-Guided Virtual Cues](https://mlsanity.com/paper/2412.02734/) (2024)
- [FocusTrack: One-Stage Focus-and-Suppress Framework for 3D Point Cloud Object Tracking](https://mlsanity.com/paper/2602.24133/) (2026)
- [SeqTrack3D: Exploring Sequence Information for Robust 3D Point Cloud Tracking](https://mlsanity.com/paper/2402.16249/) (2024)
- [Beyond Frame-wise Tracking: A Trajectory-based Paradigm for Efficient Point Cloud Tracking](https://mlsanity.com/paper/2509.11453/) (2025)
- [BEVTrack: A Simple and Strong Baseline for 3D Single Object Tracking in Bird's-Eye View](https://mlsanity.com/paper/2309.02185/) (2023)
- [FlowTrack: Point-level Flow Network for 3D Single Object Tracking](https://mlsanity.com/paper/2407.01959/) (2024)
- [Future Does Matter: Boosting 3D Object Detection with Temporal Motion Estimation in Point Cloud Sequences](https://mlsanity.com/paper/2409.04390/) (2024)
- [TFTrack: A Template-Free Framework for Efficient 3D Point Cloud Tracking](https://mlsanity.com/paper/2609.07738/) (2026)
- [Spatio-Temporal Bi-directional Cross-frame Memory for Distractor Filtering Point Cloud Single Object Tracking](https://mlsanity.com/paper/2403.15831/) (2024)
- [EasyTrack: Efficient and Compact One-stream 3D Point Clouds Tracker](https://mlsanity.com/paper/2404.05960/) (2024)

## Abstract (arXiv)

3D single object tracking with point clouds is a critical task in 3D computer vision. Previous methods usually input the last two frames and use the predicted box to get the template point cloud in previous frame and the search area point cloud in the current frame respectively, then use similarity-based or motion-based methods to predict the current box. Although these methods achieved good tracking performance, they ignore the historical information of the target, which is important for tracking. In this paper, compared to inputting two frames of point clouds, we input multi-frame of point clouds to encode the spatio-temporal information of the target and learn the motion information of the target implicitly, which could build the correlations among different frames to track the target in the current frame efficiently. Meanwhile, rather than directly using the point feature for feature fusion, we first crop the point cloud features into many patches and then use sparse attention mechanism to encode the patch-level similarity and finally fuse the multi-frame features. Extensive experiments show that our method achieves competitive results on challenging large-scale benchmarks (62.6% in KITTI and 49.66% in NuScenes).