mlsanityData to 7 Oct 2026

Skeleton-Contrastive 3D Action Representation Learning

arXiv 2108.03656 ↗First posted 8 Aug 2021cs.CVcited by 76

Not on the map. Why?

Abstract (arXiv)

This paper strives for self-supervised learning of a feature space suitable for skeleton-based action recognition. Our proposal is built upon learning invariances to input skeleton representations and various skeleton augmentations via a noise contrastive estimation. In particular, we propose inter-skeleton contrastive learning, which learns from multiple different input skeleton representations in a cross-contrastive manner. In addition, we contribute several skeleton-specific spatial and temporal augmentations which further encourage the model to learn the spatio-temporal dynamics of skeleton data. By learning similarities between different skeleton representations as well as augmented views of the same sequence, the network is encouraged to learn higher-level semantics of the skeleton data than when only using the augmented views. Our approach achieves state-of-the-art performance for self-supervised learning from skeleton data on the challenging PKU and NTU datasets with multiple downstream tasks, including action recognition, action retrieval and semi-supervised learning. Code is available at https://github.com/fmthoker/skeleton-contrast.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 8 Aug 2021.

Builds on

36 of 50 references.

References · 36

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  5. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  6. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  7. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  8. 2019Contrastive Multiview Codingcited by 1,297
  9. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  10. 2016Colorful Image Colorizationcited by 1,113
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  12. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
24 more
  1. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  2. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
  3. 2020Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognitioncited by 339
  4. 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
  5. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  6. 2020SpeedNet: Learning the Speediness in Videoscited by 171
  7. 2018Self-Supervised Video Representation Learning with Space-Time Cubic Puzzlescited by 149
  8. 2016Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognitioncited by 142
  9. 2020Memory-augmented Dense Predictive Coding for Video Representation Learningcited by 137
  10. 2016An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Datacited by 126
  11. 2020MAST: A Memory-Augmented Self-supervised Trackercited by 109
  12. 2018Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregationcited by 106
  13. 2017View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Datacited by 105
  14. 2017PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action Understandingcited by 101
  15. 2020MS2L: Multi-Task Self-Supervised Learning for Skeleton Based Action Recognitioncited by 92
  16. 2020Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognitioncited by 91
  17. 2019PREDICT & CLUSTER: Unsupervised Skeleton Based Action Recognitioncited by 90
  18. 2017Interpretable 3D Human Action Analysis with Temporal Convolutional Networkscited by 88
  19. 2020Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognitioncited by 82
  20. 2020Richly Activated Graph Convolutional Network for Robust Skeleton-based Action Recognitioncited by 36
  21. 2020Adversarial Self-Supervised Learning for Semi-Supervised 3D Action Recognitioncited by 34
  22. 2020Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglementcited by 32
  23. 2020Spatio-Temporal Inception Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 18
  24. 2020Mix Dimension in Poincar'e Geometry for 3D Skeleton-based Action Recognitioncited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 9; 2023: 27; 2024: 14; 2025: 14; 2026: 10.

76 papers.

Sort:

By area:

  1. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 188
  2. 2021Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognitioncited by 89
  3. 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
  4. 2023HYperbolic Self-Paced Learning for Self-Supervised Skeleton-based Action Representationscited by 42
  5. 2023Actionlet-Dependent Contrastive Learning for Unsupervised Skeleton-Based Action Recognitioncited by 41
  6. 2023SkeletonMAE: Graph-based Masked Autoencoder for Skeleton Sequence Pre-trainingcited by 41
  7. 2023Masked Motion Predictors are Strong 3D Action Representation Learnerscited by 41
  8. 2023Hierarchical Semantic Contrast for Scene-aware Video Anomaly Detectioncited by 39
  9. 2022CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillationcited by 38
  10. 2022Contrastive Positive Mining for Unsupervised 3D Action Representation Learningcited by 34
  11. 2022Hierarchical Consistent Contrastive Learning for Skeleton-Based Action Recognition with Growing Augmentationscited by 31
  12. 2022Hierarchical Contrast for Unsupervised Skeleton-based Action Representation Learningcited by 30
38 more
  1. 2023Language Knowledge-Assisted Representation Learning for Skeleton-Based Action Recognitioncited by 27
  2. 2023Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequencescited by 25
  3. 2023HaLP: Hallucinating Latent Positives for Skeleton-based Self-Supervised Learning of Actionscited by 25
  4. 2023Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximizationcited by 24
  5. 2022How Severe is Benchmark-Sensitivity in Video Self-Supervised Learning?cited by 23
  6. 2023Unified Multi-modal Unsupervised Representation Learning for Skeleton-based Action Understandingcited by 20
  7. 2021Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognitioncited by 19
  8. 2023Prompted Contrast with Masked Motion Modeling: Towards Versatile 3D Action Representation Learningcited by 19
  9. 2022Contrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognitioncited by 17
  10. 2023Spatiotemporal Decouple-and-Squeeze Contrastive Learning for Semi-Supervised Skeleton-based Action Recognitioncited by 16
  11. 2023Self-Supervised 3D Action Representation Learning with Skeleton Cloud Colorizationcited by 16
  12. 2024MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusioncited by 15
  13. 2024USDRL: Unified Skeleton-Based Dense Representation Learning with Multi-Grained Feature Decorrelationcited by 15
  14. 2025Heterogeneous Skeleton-Based Action Representation Learningcited by 12
  15. 2023Pyramid Self-attention Polymerization Learning for Semi-supervised Skeleton-based Action Recognitioncited by 11
  16. 2023SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-supervised Skeleton-based Action Recognitioncited by 11
  17. 2024Vision-Language Meets the Skeleton: Progressively Distillation with Cross-Modal Knowledge for 3D Action Representation Learningcited by 11
  18. 2023Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradientcited by 10
  19. 2023Skeleton-based Action Recognition through Contrasting Two-Stream Spatial-Temporal Networkscited by 9
  20. 2025SMILE: Infusing Spatial and Motion Semantics in Masked Video Learningcited by 9
  21. 2022Bootstrapped Representation Learning for Skeleton-Based Action Recognitioncited by 8
  22. 2024Identity-free Artificial Emotional Intelligence via Micro-Gesture Understandingcited by 8
  23. 2024Idempotent Unsupervised Representation Learning for Skeleton-Based Action Recognitioncited by 7
  24. 2023DMMG: Dual Min-Max Games for Self-Supervised Skeleton-Based Action Recognitioncited by 6
  25. 2023Focalized Contrastive View-invariant Learning for Self-supervised Skeleton-based Action Recognitioncited by 6
  26. 2024Expressive Keypoints for Skeleton-based Action Recognition via Skeleton Transformationcited by 6
  27. 20253D Skeleton-Based Action Recognition: A Reviewcited by 6
  28. 2024Self-Supervised Skeleton-Based Action Representation Learning: A Benchmark and Beyondcited by 5
  29. 2023Dynamic Spatial-temporal Hypergraph Convolutional Network for Skeleton-based Action Recognitioncited by 4
  30. 2025SEVERE++: Evaluating Benchmark Sensitivity in Generalization of Video Representation Learningcited by 4
  31. 2025Spatio-Temporal Joint Density Driven Learning for Skeleton-Based Action Recognitioncited by 4
  32. 2025Informative Sample Selection Model for Skeleton-based Action Recognition with Limited Training Samplescited by 4
  33. 2024STARS: Self-supervised Tuning for 3D Action Recognition in Skeleton Sequencescited by 3
  34. 2024Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluationcited by 3
  35. 2025Towards Efficient General Feature Prediction in Masked Skeleton Modelingcited by 3
  36. 2023Cross-Stream Contrastive Learning for Self-Supervised Skeleton-Based Action Recognitioncited by 2
  37. 2023Cross-Model Cross-Stream Learning for Self-Supervised Human Action Recognitioncited by 2
  38. 2024Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequencecited by 2

50 most cited of 76 All 76 (JSON)