mlsanityData to 7 Oct 2026

Spatial-Temporal Transformer for 3D Point Cloud Sequences

arXiv 2110.09783 ↗First posted 19 Oct 2021cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

Effective learning of spatial-temporal information within a point cloud sequence is highly important for many down-stream tasks such as 4D semantic segmentation and 3D action recognition. In this paper, we propose a novel framework named Point Spatial-Temporal Transformer (PST2) to learn spatial-temporal representations from dynamic 3D point cloud sequences. Our PST2 consists of two major modules: a Spatio-Temporal Self-Attention (STSA) module and a Resolution Embedding (RE) module. Our STSA module is introduced to capture the spatial-temporal context information across adjacent frames, while the RE module is proposed to aggregate features across neighbors to enhance the resolution of feature maps. We test the effectiveness our PST2 with two different tasks on point cloud sequences, i.e., 4D semantic segmentation and 3D action recognition. Extensive experiments on three benchmarks show that our PST2 outperforms existing methods on all datasets. The effectiveness of our STSA and RE modules have also been justified with ablation experiments.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 29 of its 40 references that are in the map. First posted on arXiv 19 Oct 2021.

Builds on

29 of 40 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  7. 2020Point Transformercited by 1,462
  8. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  9. 2020PCT: Point cloud transformercited by 730
  10. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  11. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  12. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
17 more
  1. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  2. 2015Multi-view Convolutional Neural Networks for 3D Shape Recognitioncited by 564
  3. 2019Deep Learning for 3D Point Clouds: A Surveycited by 557
  4. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  5. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 384
  6. 2018Video Action Transformer Networkcited by 316
  7. 2018SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filterscited by 261
  8. 2018PU-Net: Point Cloud Upsampling Networkcited by 208
  9. 2019MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequencescited by 117
  10. 2018Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Networkcited by 100
  11. 2018Recurrent Slice Networks for 3D Segmentation of Point Cloudscited by 82
  12. 2020LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attentioncited by 65
  13. 2021Learning Semantic Segmentation of Large-Scale Point Clouds with Random Samplingcited by 51
  14. 2017O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysiscited by 47
  15. 2020An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Cloudscited by 45
  16. 2019PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processingcited by 32
  17. 2020ASAP-Net: Attention and Structure Aware Point Cloud Sequence Segmentationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 9; 2024: 3; 2025: 3; 2026: 2.

22 papers.

Sort:

By area:

  1. 2022Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Cloudscited by 131
  2. 2023Geometric Clifford Algebra Networkscited by 37
  3. 2022Transformers in 3D Point Clouds: A Surveycited by 31
  4. 2022No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfacescited by 25
  5. 2023FFT-based Dynamic Token Mixer for Visioncited by 22
  6. 2023PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videoscited by 22
  7. 20223D Vision with Transformers: A Surveycited by 21
  8. 2023Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videoscited by 19
  9. 2023Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videoscited by 16
  10. 2024VG4D: Vision-Language Model Goes 4D Video Recognitioncited by 11
  11. 2022MAPLE: Masked Pseudo-Labeling autoEncoder for Semi-supervised Point Cloud Action Recognitioncited by 8
  12. 2024Dynamic 3D Point Cloud Sequences as 2D Videoscited by 7
10 more
  1. 2023Contrastive Predictive Autoencoders for Dynamic Point Cloud Self-Supervised Learningcited by 5
  2. 2023NSM4D: Neural Scene Model Based Online 4D Point Cloud Sequence Understandingcited by 5
  3. 20264DPC2hat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrappingcited by 5
  4. 2024PRENet: A Plane-Fit Redundancy Encoding Point Cloud Sequence Network for Real-Time 3D Action Recognitioncited by 2
  5. 2023Point Cloud Video Anomaly Detection Based on Point Spatio-Temporal Auto-Encodercited by 1
  6. 2023A survey on deep learning approaches for data integration in autonomous driving systemcited by 1
  7. 2025Road Boundary Detection Using 4D mmWave Radar for Autonomous Drivingcited by 1
  8. 2026SRENet: Spectral Re-Entry Network for Point Cloud Action Recognitioncited by 1
  9. 2025PDM-SSD: Single-Stage Three-Dimensional Object Detector With Point Dilationcited by 0
  10. 20253D Reconstruction from Transient Measurements with Time-Resolved Transformercited by 0