mlsanityData to 14 Oct 2026

MS-CLR: Multi-Skeleton Contrastive Learning for Human Action Recognition

arXiv 2508.14889 ↗First posted 20 Aug 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Contrastive learning has gained significant attention in skeleton-based action recognition for its ability to learn robust representations from unlabeled data. However, existing methods rely on a single skeleton convention, which limits their ability to generalize across datasets with diverse joint structures and anatomical coverage. We propose Multi-Skeleton Contrastive Learning (MS-CLR), a general self-supervised framework that aligns pose representations across multiple skeleton conventions extracted from the same sequence. This encourages the model to learn structural invariances and capture diverse anatomical cues, resulting in more expressive and generalizable features. To support this, we adapt the ST-GCN architecture to handle skeletons with varying joint layouts and scales through a unified representation scheme. Experiments on the NTU RGB+D 60 and 120 datasets demonstrate that MS-CLR consistently improves performance over strong single-skeleton contrastive learning baselines. A multi-skeleton ensemble further boosts performance, setting new state-of-the-art results on both datasets.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 20 Aug 2025.

Builds on

35 of 53 references.

References · 35

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  5. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  6. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  7. 2016Geometric deep learning: going beyond Euclidean datacited by 1,048
  8. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  9. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  10. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 403
  11. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  12. 2021Revisiting Skeleton-based Action Recognitioncited by 257
23 more
  1. 20233DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splattingcited by 195
  2. 2019An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognitioncited by 151
  3. 2017A New Representation of Skeleton Sequences for 3D Action Recognitioncited by 126
  4. 2023Drivable 3D Gaussian Avatarscited by 120
  5. 20213D Human Action Representation Learning via Cross-View Consistency Pursuitcited by 106
  6. 2021Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognitioncited by 90
  7. 2018View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognitioncited by 84
  8. 2021Skeleton-Contrastive 3D Action Representation Learningcited by 76
  9. 2022Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognitioncited by 71
  10. 2017Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networkscited by 62
  11. 2020Augmented Skeleton Based Contrastive Action Learning with Momentum LSTM for Unsupervised Action Recognitioncited by 60
  12. 2021Skeleton Cloud Colorization for Unsupervised 3D Action Representation Learningcited by 58
  13. 2017Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gatescited by 54
  14. 2022Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formatscited by 50
  15. 2023Deformable 3D Gaussian Splatting for Animatable Human Avatarscited by 44
  16. 2023Actionlet-Dependent Contrastive Learning for Unsupervised Skeleton-Based Action Recognitioncited by 42
  17. 2022CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillationcited by 38
  18. 2022Contrastive Positive Mining for Unsupervised 3D Action Representation Learningcited by 34
  19. 2022Global-local Motion Transformer for Unsupervised Skeleton-based Action Learningcited by 33
  20. 2020Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglementcited by 32
  21. 2022Hierarchical Consistent Contrastive Learning for Skeleton-Based Action Recognition with Growing Augmentationscited by 31
  22. 2022Contrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognitioncited by 17
  23. 2022Joint CNN and Transformer Network via weakly supervised Learning for efficient crowd countingcited by 6

Built on by

Not cited yet.