mlsanityData to 7 Oct 2026

PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences

arXiv 2205.13713 ↗First posted 27 May 2022cs.CVcited by 68

Not on the map. Why?

Abstract (arXiv)

Point cloud sequences are irregular and unordered in the spatial dimension while exhibiting regularities and order in the temporal dimension. Therefore, existing grid based convolutions for conventional video processing cannot be directly applied to spatio-temporal modeling of raw point cloud sequences. In this paper, we propose a point spatio-temporal (PST) convolution to achieve informative representations of point cloud sequences. The proposed PST convolution first disentangles space and time in point cloud sequences. Then, a spatial convolution is employed to capture the local structure of points in the 3D space, and a temporal convolution is used to model the dynamics of the spatial regions along the time dimension. Furthermore, we incorporate the proposed PST convolution into a deep network, namely PSTNet, to extract features of point cloud sequences in a hierarchical manner. Extensive experiments on widely-used 3D action recognition and 4D semantic segmentation datasets demonstrate the effectiveness of PSTNet to model point cloud sequences.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 39 of its 61 references that are in the map. First posted on arXiv 27 May 2022.

Builds on

39 of 61 references.

References · 39

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  6. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  7. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  8. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  9. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  10. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 899
  11. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  12. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
27 more
  1. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  2. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  3. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  4. 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
  5. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
  6. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 241
  7. 2015Beyond Short Snippets: Deep Networks for Video Classificationcited by 233
  8. 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
  9. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  10. 2018PU-Net: Point Cloud Upsampling Networkcited by 208
  11. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 208
  12. 2019An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognitioncited by 151
  13. 2019MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequencescited by 117
  14. 2018View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognitioncited by 84
  15. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  16. 2017Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networkscited by 62
  17. 2020CaSPR: Learning Canonical Spatiotemporal Point Cloud Representationscited by 51
  18. 2019SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognitioncited by 42
  19. 20203DV: 3D Dynamic Voxel for Action Recognition in Depth Videocited by 41
  20. 2019Explainable Video Action Reasoning via Prior Knowledge and State Transitionscited by 33
  21. 2018Unsupervised Learning of View-invariant Action Representationscited by 32
  22. 2019PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processingcited by 32
  23. 2019Skeleton-Based Online Action Prediction Using Scale Selection Networkcited by 20
  24. 2018Depth Pooling Based Large-scale 3D Action Recognition with Convolutional Neural Networkscited by 19
  25. 2018Action Recognition for Depth Video using Multi-view Dynamic Imagescited by 18
  26. 2019Prototype Propagation Networks (PPN) for Weakly-supervised Few-shot Learning on Category Graphcited by 15
  27. 2019Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Cloudscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 19; 2024: 16; 2025: 13; 2026: 13.

68 papers.

Sort:

By area:

  1. 2023From Chaos Comes Order: Ordering Event Representations for Object Recognition and Detectioncited by 52
  2. 2022Dynamic 3D Scene Analysis by Point Cloud Accumulationcited by 47
  3. 2023Grad-PU: Arbitrary-Scale Point Cloud Upsampling via Gradient Descent with Learned Distance Functionscited by 43
  4. 2022Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutionscited by 35
  5. 2022Point Primitive Transformer for Long-Term 4D Point Cloud Video Understandingcited by 35
  6. 2022Complete-to-Partial 4D Distillation for Self-Supervised Point Cloud Sequence Representation Learningcited by 30
  7. 2022Domain generalization of 3D semantic segmentation in autonomous drivingcited by 25
  8. 2023Clustering based Point Cloud Representation Learning for 3D Analysiscited by 23
  9. 2023PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videoscited by 22
  10. 2023Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videoscited by 19
  11. 2026Align then Adapt: Rethinking Parameter-Efficient Transfer Learning in 4D Perceptioncited by 17
  12. 2023Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videoscited by 16
38 more
  1. 2024LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernelscited by 15
  2. 2023X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-modal Knowledge Transfercited by 12
  3. 2024VG4D: Vision-Language Model Goes 4D Video Recognitioncited by 11
  4. 2024MAMBA4D: Efficient Long-Sequence Point Cloud Video Understanding with Disentangled Spatial-Temporal State Space Modelscited by 11
  5. 20233DInAction: Understanding Human Actions in 3D Point Cloudscited by 10
  6. 2022StepNet: Spatial-temporal Part-aware Network for Isolated Sign Language Recognitioncited by 9
  7. 2023Continual Learning for LiDAR Semantic Segmentation: Class-Incremental and Coarse-to-Fine strategies on Sparse Datacited by 9
  8. 2024Explore Human Parsing Modality for Action Recognitioncited by 8
  9. 2023STMT: A Spatial-Temporal Mesh Transformer for MoCap-Based Action Recognitioncited by 7
  10. 2023Multi-body SE(3) Equivariance for Unsupervised Rigid Segmentation and Motion Estimationcited by 7
  11. 2024Dynamic 3D Point Cloud Sequences as 2D Videoscited by 7
  12. 2024CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understandingcited by 6
  13. 2024Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modelingcited by 6
  14. 2024Expressive Keypoints for Skeleton-based Action Recognition via Skeleton Transformationcited by 6
  15. 2026Diffusion Masked Pretraining for Dynamic Point Cloudcited by 6
  16. 2023Learning from Mistakes: Self-Regularizing Hierarchical Representations in Point Cloud Semantic Segmentationcited by 5
  17. 2023Contrastive Predictive Autoencoders for Dynamic Point Cloud Self-Supervised Learningcited by 5
  18. 2023NSM4D: Neural Scene Model Based Online 4D Point Cloud Sequence Understandingcited by 5
  19. 2024Point2SSM++: Self-Supervised Learning of Anatomical Shape Models from Point Cloudscited by 5
  20. 20264DPC2hat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrappingcited by 5
  21. 2026Tri-Efficient Transfer Learning for Point Cloud Videoscited by 5
  22. 2025Recognizing Actions from Robotic View for Natural Human-Robot Interactioncited by 4
  23. 20254DSTR: Advancing Generative 4D Gaussians with Spatial-Temporal Rectification for High-Quality and Consistent 4D Generationcited by 4
  24. 2022Less is More: Facial Landmarks can Recognize a Spontaneous Smilecited by 3
  25. 2024A Unified Framework for Human-centric Point Cloud Video Understandingcited by 3
  26. 2024KAN-HyperpointNet for Point Cloud Sequence-Based 3D Human Action Recognitioncited by 3
  27. 2025Uni4D: A Unified Self-Supervised Learning Framework for Point Cloud Videoscited by 3
  28. 2023SpATr: MoCap 3D Human Action Recognition based on Spiral Auto-encoder and Transformer Networkcited by 2
  29. 2023DPMix: Mixture of Depth and Point Cloud Video Experts for 4D Action Segmentationcited by 2
  30. 2024PRENet: A Plane-Fit Redundancy Encoding Point Cloud Sequence Network for Real-Time 3D Action Recognitioncited by 2
  31. 2024Mask and Compress: Efficient Skeleton-based Action Recognition in Continual Learningcited by 2
  32. 2024SPiKE: 3D Human Pose from Point Cloud Sequencescited by 2
  33. 2025Content-Distortion High-Order Interaction for Blind Image Quality Assessmentcited by 2
  34. 2025sim2art: Accurate Articulated Object Modeling from a Single Video using Synthetic Training Data Onlycited by 2
  35. 2023Point Cloud Video Anomaly Detection Based on Point Spatio-Temporal Auto-Encodercited by 1
  36. 2023Generalized Robot 3D Vision-Language Model with Fast Rendering and Pre-Training Vision-Language Alignmentcited by 1
  37. 2025Quasi Zigzag Persistence: A Topological Framework for Analyzing Time-Varying Datacited by 1
  38. 2025Quality assessment of 3D human animation: Subjective and objective evaluationcited by 1

50 most cited of 68 All 68 (JSON)