mlsanityData to 7 Oct 2026

SpOT: Spatiotemporal Modeling for 3D Object Tracking

arXiv 2207.05856 ↗First posted 12 Jul 2022cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

3D multi-object tracking aims to uniquely and consistently identify all mobile entities through time. Despite the rich spatiotemporal information available in this setting, current 3D tracking methods primarily rely on abstracted information and limited history, e.g. single-frame object bounding boxes. In this work, we develop a holistic representation of traffic scenes that leverages both spatial and temporal information of the actors in the scene. Specifically, we reformulate tracking as a spatiotemporal problem by representing tracked objects as sequences of time-stamped points and bounding boxes over a long temporal history. At each timestamp, we improve the location and motion estimates of our tracked objects through learned refinement over the full sequence of object history. By considering time and space jointly, our representation naturally encodes fundamental physical priors such as object permanence and consistency across time. Our spatiotemporal tracking framework achieves state-of-the-art performance on the Waymo and nuScenes benchmarks.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 44 of its 50 references that are in the map. First posted on arXiv 12 Jul 2022.

Builds on

44 of 50 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  7. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  8. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  9. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  10. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  11. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  12. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 489
32 more
  1. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  2. 2019Tracking without bells and whistlescited by 382
  3. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  4. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  5. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  6. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  7. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  8. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
  9. 2019Disentangling Monocular 3D Object Detectioncited by 215
  10. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 202
  11. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  12. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  13. 2021Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Priorcited by 136
  14. 2020GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learningcited by 113
  15. 2021SimpleTrack: Understanding and Rethinking 3D Multi-object Trackingcited by 102
  16. 2020P2B: Point-to-Box Network for 3D Object Tracking in Point Cloudscited by 84
  17. 2020Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Drivingcited by 73
  18. 2016Vote3Deep: Fast Object Detection in 3D Point Clouds Using Efficient Convolutional Neural Networkscited by 72
  19. 2019What You See is What You Get: Exploiting Visibility for 3D Object Detectioncited by 69
  20. 2021Box-Aware Feature Enhancement for Single Object Tracking on Point Cloudscited by 66
  21. 2021PTTR: Relational 3D Point Cloud Object Tracking with Transformercited by 65
  22. 2020Probabilistic 3D Multi-Object Tracking for Autonomous Drivingcited by 64
  23. 2020AFDet: Anchor Free One Stage 3D Object Detectioncited by 63
  24. 2020AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metricscited by 63
  25. 2019Robust Multi-Modality Multi-Object Trackingcited by 56
  26. 2020CaSPR: Learning Canonical Spatiotemporal Point Cloud Representationscited by 51
  27. 20193D Multi-Object Tracking: A Baseline and New Evaluation Metricscited by 47
  28. 2020An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Cloudscited by 45
  29. 2021Learnable Online Graph Representations for 3D Multi-Object Trackingcited by 38
  30. 2021Model-free Vehicle Tracking and State Estimation in Point Cloud Sequencescited by 25
  31. 2021MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagationcited by 14
  32. 2021Online Adaptation for Implicit Object Tracking and Shape Reconstruction in the Wildcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 0; 2025: 1; 2026: 1.

10 papers.

  1. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  2. 2023Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detectioncited by 28
  3. 2022ShaSTA: Modeling Shape and Spatio-Temporal Affinities for 3D Multi-Object Trackingcited by 21
  4. 2023TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypothesescited by 14
  5. 2025CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perceptioncited by 12
  6. 2023STTracker: Spatio-Temporal Tracker for 3D Single Object Trackingcited by 10
  7. 2023TAO-Amodal: A Benchmark for Tracking Any Object Amodallycited by 7
  8. 20233D Multiple Object Tracking on Autonomous Driving: A Literature Reviewcited by 4
  9. 2026Whareformer: Learning to Track What is Where in Long Egocentric Videoscited by 2
  10. 2023ShaSTA-Fuse: Camera-LiDAR Sensor Fusion to Model Shape and Spatio-Temporal Affinities for 3D Multi-Object Trackingcited by 1