# Actionlet-Dependent Contrastive Learning for Unsupervised Skeleton-Based Action Recognition

arXiv: https://arxiv.org/abs/2303.10904 · HTML: https://mlsanity.com/paper/2303.10904/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 58 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 30 of 38)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [Unsupervised Embedding Learning via Invariant and Spreading Instance Feature](https://mlsanity.com/paper/1904.03436/) (2019)
- [An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1902.09130/) (2019)
- [Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition](https://mlsanity.com/paper/1904.01189/) (2019)
- [An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data](https://mlsanity.com/paper/1611.06067/) (2016)
- [3D Human Action Representation Learning via Cross-View Consistency Pursuit](https://mlsanity.com/paper/2104.14466/) (2021)
- [MS2L: Multi-Task Self-Supervised Learning for Skeleton Based Action Recognition](https://mlsanity.com/paper/2010.05599/) (2020)
- [PREDICT & CLUSTER: Unsupervised Skeleton Based Action Recognition](https://mlsanity.com/paper/1911.12409/) (2019)
- [Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition](https://mlsanity.com/paper/2010.09978/) (2020)
- [Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognition](https://mlsanity.com/paper/2112.03590/) (2021)
- [Skeleton-Contrastive 3D Action Representation Learning](https://mlsanity.com/paper/2108.03656/) (2021)
- [Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching](https://mlsanity.com/paper/1911.04131/) (2019)
- [Augmented Skeleton Based Contrastive Action Learning with Momentum LSTM for Unsupervised Action Recognition](https://mlsanity.com/paper/2008.00188/) (2020)
- [Skeleton Cloud Colorization for Unsupervised 3D Action Representation Learning](https://mlsanity.com/paper/2108.01959/) (2021)
- [Learning visual groups from co-occurrences in space and time](https://mlsanity.com/paper/1511.06811/) (2015)

## Built on by (most cited first, 30 of 42)

- [SkateFormer: Skeletal-Temporal Transformer for Human Action Recognition](https://mlsanity.com/paper/2403.09508/) (2024)
- [MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusion](https://mlsanity.com/paper/2409.10473/) (2024)
- [Heterogeneous Skeleton-Based Action Representation Learning](https://mlsanity.com/paper/2506.03481/) (2025)
- [SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-supervised Skeleton-based Action Recognition](https://mlsanity.com/paper/2309.05834/) (2023)
- [Vision-Language Meets the Skeleton: Progressively Distillation with Cross-Modal Knowledge for 3D Action Representation Learning](https://mlsanity.com/paper/2405.20606/) (2024)
- [Foundation Model for Skeleton-Based Human Action Understanding](https://mlsanity.com/paper/2508.12586/) (2025)
- [Identity-free Artificial Emotional Intelligence via Micro-Gesture Understanding](https://mlsanity.com/paper/2405.13206/) (2024)
- [Idempotent Unsupervised Representation Learning for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2410.20349/) (2024)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation](https://mlsanity.com/paper/2508.04513/) (2025)
- [Self-Supervised Skeleton-Based Action Representation Learning: A Benchmark and Beyond](https://mlsanity.com/paper/2406.02978/) (2024)
- [HuMoCon: Concept Discovery for Human Motion Understanding](https://mlsanity.com/paper/2505.20920/) (2025)
- [Spatio-Temporal Joint Density Driven Learning for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2505.23012/) (2025)
- [STARS: Self-supervised Tuning for 3D Action Recognition in Skeleton Sequences](https://mlsanity.com/paper/2407.10935/) (2024)
- [Towards Efficient General Feature Prediction in Masked Skeleton Modeling](https://mlsanity.com/paper/2509.03609/) (2025)
- [Skeleton2vec: A Self-supervised Learning Framework with Contextualized Target Representations for Skeleton Sequence](https://mlsanity.com/paper/2401.00921/) (2024)
- [BID: Boundary-Interior Decoding for Unsupervised Temporal Action Localization Pre-Trainin](https://mlsanity.com/paper/2403.07354/) (2024)
- [Recovering Complete Actions for Cross-dataset Skeleton Action Recognition](https://mlsanity.com/paper/2410.23641/) (2024)
- [Content-Distortion High-Order Interaction for Blind Image Quality Assessment](https://mlsanity.com/paper/2504.05076/) (2025)
- [DuoCLR: Dual-Surrogate Contrastive Learning for Skeleton-based Human Action Segmentation](https://mlsanity.com/paper/2509.05543/) (2025)
- [Attack-Augmentation Mixing-Contrastive Skeletal Representation Learning](https://mlsanity.com/paper/2304.04023/) (2023)
- [Elevating Skeleton-Based Action Recognition with Efficient Multi-Modality Self-Supervision](https://mlsanity.com/paper/2309.12009/) (2023)
- [I2MD: 3D Action Representation Learning with Inter- and Intra-modal Mutual Distillation](https://mlsanity.com/paper/2310.15568/) (2023)
- [Rehabilitation Exercise Quality Assessment through Supervised Contrastive Learning with Hard and Soft Negatives](https://mlsanity.com/paper/2403.02772/) (2024)
- [SkelMamba: A State Space Model for Efficient Skeleton Action Recognition of Neurological Disorders](https://mlsanity.com/paper/2411.19544/) (2024)
- [Multimodal Skeleton-Based Action Representation Learning via Decomposition and Composition](https://mlsanity.com/paper/2512.21064/) (2025)
- [AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild](https://mlsanity.com/paper/2605.22715/) (2026)
- [GMFL-Net: A Global Multi-geometric Feature Learning Network for Repetitive Action Counting](https://mlsanity.com/paper/2409.00330/) (2024)
- [Spatial Hierarchy and Temporal Attention Guided Cross Masking for Self-supervised Skeleton-based Action Recognition](https://mlsanity.com/paper/2409.17951/) (2024)
- [MaskSem: Semantic-Guided Masking for Learning 3D Hybrid High-Order Motion Representation](https://mlsanity.com/paper/2508.12948/) (2025)

## Abstract (arXiv)

The self-supervised pretraining paradigm has achieved great success in skeleton-based action recognition. However, these methods treat the motion and static parts equally, and lack an adaptive design for different parts, which has a negative impact on the accuracy of action recognition. To realize the adaptive action modeling of both parts, we propose an Actionlet-Dependent Contrastive Learning method (ActCLR). The actionlet, defined as the discriminative subset of the human skeleton, effectively decomposes motion regions for better action modeling. In detail, by contrasting with the static anchor without motion, we extract the motion region of the skeleton data, which serves as the actionlet, in an unsupervised manner. Then, centering on actionlet, a motion-adaptive data transformation method is built. Different data transformations are applied to actionlet and non-actionlet regions to introduce more diversity while maintaining their own characteristics. Meanwhile, we propose a semantic-aware feature pooling method to build feature representations among motion and static regions in a distinguished manner. Extensive experiments on NTU RGB+D and PKUMMD show that the proposed method achieves remarkable action recognition performance. More visualization and quantitative experiments demonstrate the effectiveness of our method. Our project website is available at https://langlandslin.github.io/projects/ActCLR/