mlsanityData to 7 Oct 2026

Skeleton Cloud Colorization for Unsupervised 3D Action Representation Learning

arXiv 2108.01959 ↗First posted 4 Aug 2021cs.CVcited by 58

Not on the map. Why?

Abstract (arXiv)

Skeleton-based human action recognition has attracted increasing attention in recent years. However, most of the existing works focus on supervised learning which requiring a large number of annotated action sequences that are often expensive to collect. We investigate unsupervised representation learning for skeleton action recognition, and design a novel skeleton cloud colorization technique that is capable of learning skeleton representations from unlabeled skeleton sequence data. Specifically, we represent a skeleton action sequence as a 3D skeleton cloud and colorize each point in the cloud according to its temporal and spatial orders in the original (unannotated) skeleton sequence. Leveraging the colorized skeleton point cloud, we design an auto-encoder framework that can learn spatial-temporal features from the artificial color labels of skeleton joints effectively. We evaluate our skeleton cloud colorization approach with action classifiers trained under different configurations, including unsupervised, semi-supervised and fully-supervised settings. Extensive experiments on NTU RGB+D and NW-UCLA datasets show that the proposed method outperforms existing unsupervised and semi-supervised 3D action recognition methods by large margins, and it achieves competitive performance in supervised 3D action recognition as well.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). Builds on 34 of its 58 references that are in the map. First posted on arXiv 4 Aug 2021.

Builds on

34 of 58 references.

References · 34

  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  3. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  4. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  5. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  6. 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
  7. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  8. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  9. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  10. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  11. 2017Temporal Segment Networks for Action Recognition in Videoscited by 279
  12. 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
22 more
  1. 2019An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognitioncited by 151
  2. 2016Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognitioncited by 142
  3. 2014Cross-view Action Modeling, Learning and Recognitioncited by 134
  4. 2017A New Representation of Skeleton Sequences for 3D Action Recognitioncited by 125
  5. 2017View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Datacited by 105
  6. 2019Skeleton-Based Action Recognition with Multi-Stream Adaptive Graph Convolutional Networkscited by 103
  7. 2021UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehiclescited by 95
  8. 2020MS2L: Multi-Task Self-Supervised Learning for Skeleton Based Action Recognitioncited by 92
  9. 2019PREDICT & CLUSTER: Unsupervised Skeleton Based Action Recognitioncited by 90
  10. 2018View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognitioncited by 84
  11. 2019Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searchingcited by 70
  12. 2017Skeleton-based Action Recognition with Convolutional Neural Networkscited by 54
  13. 2017Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gatescited by 54
  14. 20203DV: 3D Dynamic Voxel for Action Recognition in Depth Videocited by 41
  15. 2017Unsupervised Learning of Long-Term Motion Dynamics for Videoscited by 37
  16. 2020Adversarial Self-Supervised Learning for Semi-Supervised 3D Action Recognitioncited by 34
  17. 2018Unsupervised Learning of View-invariant Action Representationscited by 32
  18. 2018Unsupervised Feature Learning of Human Actions as Trajectories in Pose Embedding Manifoldcited by 26
  19. 2017Scene Flow to Action Map: A New Representation for RGB-D based Action Recognition with Convolutional Neural Networkscited by 20
  20. 2014HOPC: Histogram of Oriented Principal Components of 3D Pointclouds for Action Recognitioncited by 15
  21. 2020Sparse Semi-Supervised Action Recognition with Active Learningcited by 6
  22. 2017YoTube: Searching Action Proposal via Recurrent and Static Regression Networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 11; 2023: 22; 2024: 11; 2025: 6; 2026: 5.

58 papers.

Sort:

  1. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 188
  2. 2021Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognitioncited by 89
  3. 2023HYperbolic Self-Paced Learning for Self-Supervised Skeleton-based Action Representationscited by 42
  4. 2023Actionlet-Dependent Contrastive Learning for Unsupervised Skeleton-Based Action Recognitioncited by 41
  5. 2023SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-trainingcited by 41
  6. 2023Masked Motion Predictors are Strong 3D Action Representation Learnerscited by 41
  7. 2022CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillationcited by 38
  8. 2022Global-local Motion Transformer for Unsupervised Skeleton-based Action Learningcited by 33
  9. 2022Hierarchical Consistent Contrastive Learning for Skeleton-Based Action Recognition with Growing Augmentationscited by 31
  10. 2022Hierarchically Self-Supervised Transformer for Human Skeleton Representation Learningcited by 30
  11. 2022Hierarchical Contrast for Unsupervised Skeleton-based Action Representation Learningcited by 30
  12. 2023HaLP: Hallucinating Latent Positives for Skeleton-based Self-Supervised Learning of Actionscited by 25
38 more
  1. 2022ViA: View-invariant Skeleton Action Representation Learning via Motion Retargetingcited by 22
  2. 2023Hulk: A Universal Knowledge Translator for Human-Centric Taskscited by 22
  3. 2021ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpointscited by 20
  4. 2021Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognitioncited by 19
  5. 2022Contrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognitioncited by 17
  6. 2023Spatiotemporal Decouple-and-Squeeze Contrastive Learning for Semi-Supervised Skeleton-based Action Recognitioncited by 16
  7. 2023Self-Supervised 3D Action Representation Learning with Skeleton Cloud Colorizationcited by 16
  8. 2023LAC: Latent Action Composition for Skeleton-based Action Segmentationcited by 15
  9. 2025Heterogeneous Skeleton-Based Action Representation Learningcited by 12
  10. 2023Pyramid Self-attention Polymerization Learning for Semi-supervised Skeleton-based Action Recognitioncited by 11
  11. 2023SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-supervised Skeleton-based Action Recognitioncited by 11
  12. 2024Vision-Language Meets the Skeleton: Progressively Distillation with Cross-Modal Knowledge for 3D Action Representation Learningcited by 11
  13. 20233DInAction: Understanding Human Actions in 3D Point Cloudscited by 10
  14. 2024Identity-free Artificial Emotional Intelligence via Micro-Gesture Understandingcited by 8
  15. 2024Towards Physical World Backdoor Attacks against Skeleton Action Recognitioncited by 7
  16. 2024Idempotent Unsupervised Representation Learning for Skeleton-Based Action Recognitioncited by 7
  17. 2023DMMG: Dual Min-Max Games for Self-Supervised Skeleton-Based Action Recognitioncited by 6
  18. 2023Focalized Contrastive View-invariant Learning for Self-supervised Skeleton-based Action Recognitioncited by 6
  19. 2024MV2MAE: Multi-View Video Masked Autoencoderscited by 6
  20. 2024HumMUSS: Human Motion Understanding using State Space Modelscited by 6
  21. 2023Contrastive Predictive Autoencoders for Dynamic Point Cloud Self-Supervised Learningcited by 5
  22. 2024Self-Supervised Skeleton-Based Action Representation Learning: A Benchmark and Beyondcited by 5
  23. 2025MoFM: A Large-Scale Human Motion Foundation Modelcited by 5
  24. 2022Towards To-a-T Spatio-Temporal Focus for Skeleton-Based Action Recognitioncited by 4
  25. 2022Collaborating Domain-shared and Target-specific Feature Clustering for Cross-domain 3D Action Recognitioncited by 4
  26. 2023Dynamic Spatial-temporal Hypergraph Convolutional Network for Skeleton-based Action Recognitioncited by 4
  27. 2023Token Boosting for Robust Self-Supervised Visual Transformer Pre-trainingcited by 4
  28. 2024Dynamic Spatial-Temporal Aggregation for Skeleton-Aware Sign Language Recognitioncited by 4
  29. 2025Spatio-Temporal Joint Density Driven Learning for Skeleton-Based Action Recognitioncited by 4
  30. 2022AFE-CNN: 3D Skeleton-based Action Recognition with Action Feature Enhancementcited by 3
  31. 2024STARS: Self-supervised Tuning for 3D Action Recognition in Skeleton Sequencescited by 3
  32. 2025Towards Efficient General Feature Prediction in Masked Skeleton Modelingcited by 3
  33. 2023Cross-Stream Contrastive Learning for Self-Supervised Skeleton-Based Action Recognitioncited by 2
  34. 2023Cross-Model Cross-Stream Learning for Self-Supervised Human Action Recognitioncited by 2
  35. 2024Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequencecited by 2
  36. 2023Attack-Augmentation Mixing-Contrastive Skeletal Representation Learningcited by 1
  37. 2023Cross-view Action Recognition via Contrastive View-invariant Representationcited by 1
  38. 2023I2MD: 3D Action Representation Learning with Inter- and Intra-modal Mutual Distillationcited by 1

50 most cited of 58 All 58 (JSON)