mlsanityData to 14 Oct 2026

A Unified Framework for Human-centric Point Cloud Video Understanding

arXiv 2403.20031 ↗First posted 29 Mar 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Human-centric Point Cloud Video Understanding (PVU) is an emerging field focused on extracting and interpreting human-related features from sequences of human point clouds, further advancing downstream human-centric tasks and applications. Previous works usually focus on tackling one specific task and rely on huge labeled data, which has poor generalization capability. Considering that human has specific characteristics, including the structural semantics of human body and the dynamics of human motions, we propose a unified framework to make full use of the prior knowledge and explore the inherent features in the data itself for generalized human-centric point cloud video understanding. Extensive experiments demonstrate that our method achieves state-of-the-art performance on various human-related tasks, including action recognition and 3D pose estimation. All datasets and code will be released soon.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 41 of its 50 references that are in the map. First posted on arXiv 29 Mar 2024.

Builds on

41 of 50 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  6. 2020Point Transformercited by 1,470
  7. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  9. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  10. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  11. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  12. 2020PCT: Point cloud transformercited by 732
29 more
  1. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  2. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  3. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  4. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 436
  5. 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 379
  6. 2017Learning from Synthetic Humanscited by 359
  7. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 264
  8. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 242
  9. 2020FLOT: Scene Flow on Point Clouds Guided by Optimal Transportcited by 129
  10. 2019MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequencescited by 117
  11. 2023PointGPT: Auto-regressively Generative Pre-training from Point Cloudscited by 115
  12. 2019HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Cloudscited by 109
  13. 2022CenterFormer: Center-based Transformer for 3D Object Detectioncited by 99
  14. 2022PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequencescited by 68
  15. 2021Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perceptioncited by 57
  16. 2023SLOPER4D: A Scene-Aware Dataset for Global 4D Human Pose Estimation in Urban Environmentscited by 47
  17. 2022LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensorscited by 44
  18. 2019PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Cloudscited by 38
  19. 2022Point Primitive Transformer for Long-Term 4D Point Cloud Video Understandingcited by 35
  20. 2022Complete-to-Partial 4D Distillation for Self-Supervised Point Cloud Sequence Representation Learningcited by 30
  21. 2023CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactionscited by 27
  22. 2022Weakly Supervised 3D Multi-person Pose Estimation for Large-scale Scenes based on Monocular Camera and Single LiDARcited by 23
  23. 2023PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videoscited by 22
  24. 2023Point Cloud Classification Using Content-based Transformer via Clustering in Feature Spacecited by 19
  25. 2023Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videoscited by 19
  26. 2023Human-centric Scene Understanding for 3D Large-scale Scenarioscited by 17
  27. 2022Gait Recognition in Large-scale Free Environment via Single LiDARcited by 16
  28. 2023Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videoscited by 16
  29. 2021Input-Output Balanced Framework for Long-tailed LiDAR Semantic Segmentationcited by 8

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2025PUMPS: Skeleton-Agnostic Point-based Universal Motion Pre-Training for Synthesis in Human Motion Taskscited by 3
  2. 2024Towards Practical Human Motion Prediction with LiDAR Point Cloudscited by 2
  3. 20253D Human Pose and Shape Estimation from LiDAR Point Clouds: A Reviewcited by 0