# Fusing Higher-order Features in Graph Neural Networks for Skeleton-based Action Recognition

arXiv: https://arxiv.org/abs/2105.01563 · HTML: https://mlsanity.com/paper/2105.01563/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 4 May 2021.

## Builds on (most cited first, 29 of 29)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [Word-level Deep Sign Language Recognition from Video: A New Large-scale Dataset and Methods Comparison](https://mlsanity.com/paper/1910.11006/) (2019)
- [An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1902.09130/) (2019)
- [Semantics-Guided Neural Networks for Efficient Skeleton-Based Human Action Recognition](https://mlsanity.com/paper/1904.01189/) (2019)
- [Dual Path Networks](https://mlsanity.com/paper/1707.01629/) (2017)
- [Co-occurrence Feature Learning from Skeleton Data for Action Recognition and Detection with Hierarchical Aggregation](https://mlsanity.com/paper/1804.06055/) (2018)
- [Invertible Denoising Network: A Light Solution for Real Noise Removal](https://mlsanity.com/paper/2104.10546/) (2021)
- [Transferring Cross-domain Knowledge for Video Sign Language Recognition](https://mlsanity.com/paper/2003.03703/) (2020)
- [Densely Residual Laplacian Super-Resolution](https://mlsanity.com/paper/1906.12021/) (2019)
- [Scattering GCN: Overcoming Oversmoothness in Graph Convolutional Networks](https://mlsanity.com/paper/2003.08414/) (2020)
- [Open Set Domain Adaptation: Theoretical Bound and Algorithm](https://mlsanity.com/paper/1907.08375/) (2019)
- [Memory Attention Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1804.08254/) (2018)
- [A Comparative Review of Recent Kinect-based Action Recognition Algorithms](https://mlsanity.com/paper/1906.09955/) (2019)
- [Tensor Representations for Action Recognition](https://mlsanity.com/paper/2012.14371/) (2020)
- [BASAR:Black-box Attack on Skeletal Action Recognition](https://mlsanity.com/paper/2103.05266/) (2021)
- [Rethinking Softmax with Cross-Entropy: Neural Network Classifier as Mutual Information Estimator](https://mlsanity.com/paper/1911.10688/) (2019)
- [Understanding the Robustness of Skeleton-based Action Recognition under Adversarial Attack](https://mlsanity.com/paper/2103.05347/) (2021)
- [Skeleton-DML: Deep Metric Learning for Skeleton-Based One-Shot Action Recognition](https://mlsanity.com/paper/2012.13823/) (2020)
- [AdaSGN: Adapting Joint Number and Model Size for Efficient Skeleton-Based Action Recognition](https://mlsanity.com/paper/2103.11770/) (2021)
- [Fast Haar Transforms for Graph Neural Networks](https://mlsanity.com/paper/1907.04786/) (2019)
- [How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?](https://mlsanity.com/paper/2101.01104/) (2020)
- [Tensor Representations via Kernel Linearization for Action Recognition from 3D Skeletons (Extended Version)](https://mlsanity.com/paper/1604.00239/) (2016)
- [Clarinet: A One-step Approach Towards Budget-friendly Unsupervised Domain Adaptation](https://mlsanity.com/paper/2007.14612/) (2020)

## Built on by (most cited first, 22 of 22)

- [Advancing Video Anomaly Detection: A Concise Review and a New Dataset](https://mlsanity.com/paper/2402.04857/) (2024)
- [3Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognition](https://mlsanity.com/paper/2303.14474/) (2023)
- [Temporal-Viewpoint Transportation Plan for Skeletal Few-shot Action Recognition](https://mlsanity.com/paper/2210.16820/) (2022)
- [High-order Tensor Pooling with Attention for Action Recognition](https://mlsanity.com/paper/2110.05216/) (2021)
- [Skeleton-based Action Recognition via Temporal-Channel Aggregation](https://mlsanity.com/paper/2205.15936/) (2022)
- [Flow Dynamics Correction for Action Recognition](https://mlsanity.com/paper/2310.10059/) (2023)
- [Taylor Videos for Action Recognition](https://mlsanity.com/paper/2402.03019/) (2024)
- [STEP CATFormer: Spatial-Temporal Effective Body-Part Cross Attention Transformer for Skeleton-based Action Recognition](https://mlsanity.com/paper/2312.03288/) (2023)
- [Meet JEANIE: a Similarity Measure for 3D Skeleton Sequences via Temporal-Viewpoint Alignment](https://mlsanity.com/paper/2402.04599/) (2024)
- [StepNet: Spatial-temporal Part-aware Network for Isolated Sign Language Recognition](https://mlsanity.com/paper/2212.12857/) (2022)
- [Ske2Grid: Skeleton-to-Grid Representation Learning for Action Recognition](https://mlsanity.com/paper/2308.07571/) (2023)
- [TSGCNeXt: Dynamic-Static Multi-Graph Convolution for Efficient Skeleton-Based Action Recognition with Long-term Learning Potential](https://mlsanity.com/paper/2304.11631/) (2023)
- [Towards To-a-T Spatio-Temporal Focus for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2202.02314/) (2022)
- [Shifting Perspective to See Difference: A Novel Multi-View Method for Skeleton based Action Recognition](https://mlsanity.com/paper/2209.02986/) (2022)
- [Dynamic Spatial-temporal Hypergraph Convolutional Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2302.08689/) (2023)
- [Using joint angles based on the international biomechanical standards for human action recognition and related tasks](https://mlsanity.com/paper/2406.17443/) (2024)
- [Feature Hallucination for Self-supervised Action Recognition](https://mlsanity.com/paper/2506.20342/) (2025)
- [Self-attention based anchor proposal for skeleton-based action recognition](https://mlsanity.com/paper/2112.09413/) (2021)
- [Multi-Dimensional Refinement Graph Convolutional Network with Robust Decouple Loss for Fine-Grained Skeleton-Based Action Recognition](https://mlsanity.com/paper/2306.15321/) (2023)
- [Local Spherical Harmonics Improve Skeleton-Based Hand Action Recognition](https://mlsanity.com/paper/2308.10557/) (2023)
- [Symplectic Structure-Aware Hamiltonian (Graph) Embeddings](https://mlsanity.com/paper/2309.04885/) (2023)
- [Human Action Recognition (HAR) Using Skeleton-based Spatial Temporal Relative Transformer Network: ST-RTR](https://mlsanity.com/paper/2410.23806/) (2024)

## Abstract (arXiv)

Skeleton sequences are lightweight and compact, and thus are ideal candidates for action recognition on edge devices. Recent skeleton-based action recognition methods extract features from 3D joint coordinates as spatial-temporal cues, using these representations in a graph neural network for feature fusion to boost recognition performance. The use of first- and second-order features, i.e., joint and bone representations, has led to high accuracy. Nonetheless, many models are still confused by actions that have similar motion trajectories. To address these issues, we propose fusing higher-order features in the form of angular encoding into modern architectures to robustly capture the relationships between joints and body parts. This simple fusion with popular spatial-temporal graph neural networks achieves new state-of-the-art accuracy in two large benchmarks, including NTU60 and NTU120, while employing fewer parameters and reduced run time. Our source code is publicly available at: https://github.com/ZhenyueQin/Angular-Skeleton-Encoding.