# Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition

arXiv: https://arxiv.org/abs/2411.18941 · HTML: https://mlsanity.com/paper/2411.18941/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 46 references that are in the map.
- First posted on arXiv 28 Nov 2024.

## Builds on (most cited first, 30 of 37)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment](https://mlsanity.com/paper/1908.06391/) (2019)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2107.12213/) (2021)
- [Revisiting Skeleton-based Action Recognition](https://mlsanity.com/paper/2104.13586/) (2021)
- [FineGym: A Hierarchical Video Dataset for Fine-grained Action Understanding](https://mlsanity.com/paper/2004.06704/) (2020)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [Prototype Mixture Models for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2008.03898/) (2020)
- [Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition](https://mlsanity.com/paper/1904.01189/) (2019)
- [Robust Classification with Convolutional Prototype Learning](https://mlsanity.com/paper/1805.03438/) (2018)
- [3D Human Action Representation Learning via Cross-View Consistency Pursuit](https://mlsanity.com/paper/2104.14466/) (2021)
- [Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognition](https://mlsanity.com/paper/2112.03590/) (2021)
- [Hierarchically Decomposed Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2208.10741/) (2022)
- [PYSKL: Towards Good Practices for Skeleton Action Recognition](https://mlsanity.com/paper/2205.09443/) (2022)
- [Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2206.13028/) (2022)
- [Learning Discriminative Representations for Skeleton Based Action Recognition](https://mlsanity.com/paper/2303.03729/) (2023)
- [Generative Action Description Prompts for Skeleton-based Action Recognition](https://mlsanity.com/paper/2208.05318/) (2022)

## Built on by (most cited first, 29 of 29)

- [SkeletonContext: Skeleton-side Context Prompt Learning for Zero-Shot Skeleton-based Action Recognition](https://mlsanity.com/paper/2603.29692/) (2026)
- [BST: Badminton Stroke-type Transformer for Skeleton-based Action Recognition in Racket Sports](https://mlsanity.com/paper/2502.21085/) (2025)
- [LSTC-MDA: A Unified Framework for Long-Short Term Temporal Convolution and Mixed Data Augmentation in Skeleton-Based Action Recognition](https://mlsanity.com/paper/2509.14619/) (2025)
- [SkeletonAgent: An Agentic Interaction Framework for Skeleton-based Action Recognition](https://mlsanity.com/paper/2511.22433/) (2025)
- [Heatmap Pooling Network for Action Recognition from RGB Videos](https://mlsanity.com/paper/2512.03837/) (2025)
- [Synchronized and Fine-Grained Head for Skeleton-Based Ambiguous Action Recognition](https://mlsanity.com/paper/2412.14833/) (2024)
- [Variational Graph Convolutional Neural Networks](https://mlsanity.com/paper/2507.01699/) (2025)
- [MMeViT: Multi-Modal ensemble ViT for Post-Stroke Rehabilitation Action Recognition](https://mlsanity.com/paper/2509.23044/) (2025)
- [Label-Efficient Skeleton-based Recognition with Stable-Invertible Graph Convolutional Networks](https://mlsanity.com/paper/2511.17345/) (2025)
- [Active Learning for GCN-based Action Recognition](https://mlsanity.com/paper/2511.21625/) (2025)
- [Skeleton-Snippet Contrastive Learning with Multiscale Feature Fusion for Action Localization](https://mlsanity.com/paper/2512.16504/) (2025)
- [Patch as Node: Human-Centric Graph Representation Learning for Multimodal Action Recognition](https://mlsanity.com/paper/2512.21916/) (2025)
- [Affinity Contrastive Learning for Skeleton-based Human Activity Understanding](https://mlsanity.com/paper/2601.16694/) (2026)
- [Skarimva: Skeleton-based Action Recognition is a Multi-view Application](https://mlsanity.com/paper/2602.23231/) (2026)
- [E2E-GNet: An End-to-End Skeleton-based Geometric Deep Neural Network for Human Motion Recognition](https://mlsanity.com/paper/2603.02477/) (2026)
- [SBF: An Effective Representation to Augment Skeleton for Video-based Human Action Recognition](https://mlsanity.com/paper/2604.03590/) (2026)
- [Toward Universal Skeleton-Based Action Recognition across Heterogeneous Skeletons and Open Vocabularies](https://mlsanity.com/paper/2604.17013/) (2026)
- [Beyond Binary Contrast: Modeling Continuous Skeleton Action Spaces with Transitional Anchors](https://mlsanity.com/paper/2604.17914/) (2026)
- [Action Motifs: Self-Supervised Hierarchical Representation of Human Body Movements](https://mlsanity.com/paper/2604.28173/) (2026)
- [High-Speed Vision Improves Zero-Shot Semantic Understanding of Human Actions](https://mlsanity.com/paper/2605.00496/) (2026)
- [DAP: Doppler-aware Point Network for Heterogeneous mmWave Action Recognition](https://mlsanity.com/paper/2605.09604/) (2026)
- [Improving Viewpoint-Invariance and Temporal Consistency for Action Detection](https://mlsanity.com/paper/2605.22695/) (2026)
- [Cross-Domain Human Action Recognition from Multiview Motion and Textual Descriptions](https://mlsanity.com/paper/2605.22697/) (2026)
- [SkelHCC: A Hyperbolic CLIP-Driven Cache Adaptation Framework for Skeleton-based One-Shot Action Recognition](https://mlsanity.com/paper/2606.03610/) (2026)
- [Partial Skeleton Visibility for Action Recognition: A Constrained Field-of-View Approach](https://mlsanity.com/paper/2607.00716/) (2026)
- [Capturing Uncertainty in Human Motion for Representation Learning in Soccer](https://mlsanity.com/paper/2608.11203/) (2026)
- [Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition](https://mlsanity.com/paper/2609.02510/) (2026)
- [SoftRerank: Hierarchical Soft Fusion with Candidate-Label Reranking for Long-Tailed Micro-Action Recognition](https://mlsanity.com/paper/2609.08221/) (2026)
- [PhysioAI: Clinical Knowledge-Guided Semantic Supervision for Skeleton-Based Physiotherapy Action Recognition](https://mlsanity.com/paper/2609.12491/) (2026)

## Abstract (arXiv)

In skeleton-based action recognition, a key challenge is distinguishing between actions with similar trajectories of joints due to the lack of image-level details in skeletal representations. Recognizing that the differentiation of similar actions relies on subtle motion details in specific body parts, we direct our approach to focus on the fine-grained motion of local skeleton components. To this end, we introduce ProtoGCN, a Graph Convolutional Network (GCN)-based model that breaks down the dynamics of entire skeleton sequences into a combination of learnable prototypes representing core motion patterns of action units. By contrasting the reconstruction of prototypes, ProtoGCN can effectively identify and enhance the discriminative representation of similar actions. Without bells and whistles, ProtoGCN achieves state-of-the-art performance on multiple benchmark datasets, including NTU RGB+D, NTU RGB+D 120, Kinetics-Skeleton, and FineGYM, which demonstrates the effectiveness of the proposed method. The code is available at https://github.com/firework8/ProtoGCN.