mlsanityData to 7 Oct 2026

Multi-Scale Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition

arXiv 2111.03993 ↗First posted 7 Nov 2021cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Skeleton data is of low dimension. However, there is a trend of using very deep and complicated feedforward neural networks to model the skeleton sequence without considering the complexity in recent year. In this paper, a simple yet effective multi-scale semantics-guided neural network (MS-SGN) is proposed for skeleton-based action recognition. We explicitly introduce the high level semantics of joints (joint type and frame index) into the network to enhance the feature representation capability of joints. Moreover, a multi-scale strategy is proposed to be robust to the temporal scale variations. In addition, we exploit the relationship of joints hierarchically through two modules, i.e., a joint-level module for modeling the correlations of joints in the same frame and a frame-level module for modeling the temporal dependencies of frames. With an order of magnitude smaller model size than most previous methods, MSSGN achieves the state-of-the-art performance on the NTU60, NTU120, and SYSU datasets.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 40 of its 66 references that are in the map. First posted on arXiv 7 Nov 2021.

Builds on

40 of 66 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  8. 2017Non-local Neural Networkscited by 2,290
  9. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  10. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  11. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  12. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
28 more
  1. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  2. 2017Convolutional Sequence to Sequence Learningcited by 550
  3. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
  4. 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 430
  5. 2020Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognitioncited by 339
  6. 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
  7. 2018Videos as Space-Time Region Graphscited by 221
  8. 2019An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognitioncited by 151
  9. 2016Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognitioncited by 142
  10. 2019Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognitioncited by 135
  11. 2016An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Datacited by 126
  12. 2017A New Representation of Skeleton Sequences for 3D Action Recognitioncited by 125
  13. 2018Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregationcited by 106
  14. 2017View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Datacited by 105
  15. 2018View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognitioncited by 84
  16. 2016Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networkscited by 63
  17. 2017Skeleton-Based Human Action Recognition with Global Context-Aware Attention LSTM Networkscited by 62
  18. 2017Skeleton-based Action Recognition with Convolutional Neural Networkscited by 54
  19. 2017Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gatescited by 54
  20. 2019Learning Deep Bilinear Transformation for Fine-grained Image Representationcited by 54
  21. 2017The Microsoft 2017 Conversational Speech Recognition Systemcited by 51
  22. 2018Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learningcited by 47
  23. 2017RGB-D-based Human Motion Recognition with Deep Learning: A Surveycited by 37
  24. 2018Glimpse Clouds: Human Activity Recognition from Unstructured Feature Pointscited by 33
  25. 2016Space-Time Representation of People Based on 3D Skeletal Data: A Reviewcited by 24
  26. 2016Transition Forests: Learning Discriminative Temporal Transitions for Action Recognition and Detectioncited by 18
  27. 2018Adding Attentiveness to the Neurons in Recurrent Neural Networkscited by 11
  28. 2019EleAtt-RNN: Adding Attentiveness to Neurons in Recurrent Neural Networkscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024AutoGCN -- Towards Generic Human Activity Recognition with Neural Architecture Searchcited by 3
  2. 2026Skeleton-to-Image Encoding: Enabling Skeleton Representation Learning via Vision-Pretrained Modelscited by 0