mlsanityData to 14 Oct 2026

Human Action Recognition from Point Clouds over Time

arXiv 2510.05506 ↗First posted 7 Oct 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent research into human action recognition (HAR) has focused predominantly on skeletal action recognition and video-based methods. With the increasing availability of consumer-grade depth sensors and Lidar instruments, there is a growing opportunity to leverage dense 3D data for action recognition, to develop a third way. This paper presents a novel approach for recognizing actions from 3D videos by introducing a pipeline that segments human point clouds from the background of a scene, tracks individuals over time, and performs body part segmentation. The method supports point clouds from both depth sensors and monocular depth estimation. At the core of the proposed HAR framework is a novel backbone for 3D action recognition, which combines point-based techniques with sparse convolutional networks applied to voxel-mapped point cloud sequences. Experiments incorporate auxiliary point features including surface normals, color, infrared intensity, and body part parsing labels, to enhance recognition accuracy. Evaluation on the NTU RGB- D 120 dataset demonstrates that the method is competitive with existing skeletal action recognition algorithms. Moreover, combining both sensor-based and estimated depth inputs in an ensemble setup, this approach achieves 89.3% accuracy when different human subjects are considered for training and testing, outperforming previous point cloud action recognition methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 47 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

36 of 47 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  8. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  9. 2018SlowFast Networks for Video Recognitioncited by 1,867
  10. 2024Depth Anything V2cited by 1,422
  11. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  12. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
24 more
  1. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  2. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  3. 2021Multiscale Vision Transformerscited by 897
  4. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  5. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  6. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  7. 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
  8. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  9. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 388
  10. 2017Submanifold Sparse Convolutional Networkscited by 223
  11. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 200
  12. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  13. 2022Focal Sparse Convolutional Networks for 3D Object Detectioncited by 142
  14. 2017Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsingcited by 109
  15. 2023OctFormer: Octree-based Transformers for 3D Point Cloudscited by 109
  16. 2022PYSKL: Towards Good Practices for Skeleton Action Recognitioncited by 83
  17. 2022PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequencescited by 68
  18. 2018PoseFix: Model-agnostic General Human Pose Refinement Networkcited by 47
  19. 20203DV: 3D Dynamic Voxel for Action Recognition in Depth Videocited by 41
  20. 2022Transformers in 3D Point Clouds: A Surveycited by 31
  21. 2015Rank Pooling for Action Recognitioncited by 29
  22. 2023Deep Learning Technique for Human Parsing: A Survey and Outlookcited by 18
  23. 20233DInAction: Understanding Human Actions in 3D Point Cloudscited by 10
  24. 2022RALACs: Action Recognition in Autonomous Vehicles using Interaction Encoding and Optical Flowcited by 2

Built on by

Not cited yet.