# Hierarchical Graph Convolutional Skeleton Transformer for Action Recognition

arXiv: https://arxiv.org/abs/2109.02860 · HTML: https://mlsanity.com/paper/2109.02860/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 16 references that are in the map.
- First posted on arXiv 7 Sep 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [LocalViT: Analyzing Locality in Vision Transformers](https://mlsanity.com/paper/2104.05707/) (2021)
- [Stronger, Faster and More Explainable: A Graph Convolutional Baseline for Skeleton-based Action Recognition](https://mlsanity.com/paper/2010.09978/) (2020)
- [Dynamic GCN: Context-enriched Topology Learning for Skeleton-based Action Recognition](https://mlsanity.com/paper/2007.14690/) (2020)
- [Spatial Temporal Transformer Network for Skeleton-based Action Recognition](https://mlsanity.com/paper/2012.06399/) (2020)

## Built on by (most cited first, 12 of 12)

- [Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformer](https://mlsanity.com/paper/2407.12322/) (2024)
- [Delving Deep into One-Shot Skeleton-based Action Recognition with Diverse Occlusions](https://mlsanity.com/paper/2202.11423/) (2022)
- [Graph Convolutional Networks for Multi-modality Medical Imaging: Methods, Architectures, and Clinical Applications](https://mlsanity.com/paper/2202.08916/) (2022)
- [3D Skeleton-Based Action Recognition: A Review](https://mlsanity.com/paper/2506.00915/) (2025)
- [HDBN: A Novel Hybrid Dual-branch Network for Robust Skeleton-based Action Recognition](https://mlsanity.com/paper/2404.15719/) (2024)
- [HyLiFormer: Hyperbolic Linear Attention for Skeleton-based Human Action Recognition](https://mlsanity.com/paper/2502.05869/) (2025)
- [DD-GCN: Directed Diffusion Graph Convolutional Network for Skeleton-based Human Action Recognition](https://mlsanity.com/paper/2308.12501/) (2023)
- [Signal-SGN: A Spiking Graph Convolutional Network for Skeletal Action Recognition via Learning Temporal-Frequency Dynamics](https://mlsanity.com/paper/2408.01701/) (2024)
- [Pose Uncertainty Aware Movement Synchrony Estimation via Spatial-Temporal Graph Transformer](https://mlsanity.com/paper/2208.01161/) (2022)
- [FreqMixFormerV2: Lightweight Frequency-aware Mixed Transformer for Human Skeleton Action Recognition](https://mlsanity.com/paper/2412.20621/) (2024)
- [GvT: A Graph-based Vision Transformer with Talking-Heads Utilizing Sparsity, Trained from Scratch on Small Datasets](https://mlsanity.com/paper/2404.04924/) (2024)
- [Disentangling Genotype and Environment Specific Latent Features for Improved Trait Prediction using a Compositional Autoencoder](https://mlsanity.com/paper/2410.19922/) (2024)

## Abstract (arXiv)

Graph convolutional networks (GCNs) have emerged as dominant methods for skeleton-based action recognition. However, they still suffer from two problems, namely, neighborhood constraints and entangled spatiotemporal feature representations. Most studies have focused on improving the design of graph topology to solve the first problem but they have yet to fully explore the latter. In this work, we design a disentangled spatiotemporal transformer (DSTT) block to overcome the above limitations of GCNs in three steps: (i) feature disentanglement for spatiotemporal decomposition;(ii) global spatiotemporal attention for capturing correlations in the global context; and (iii) local information enhancement for utilizing more local information. Thereon, we propose a novel architecture, named Hierarchical Graph Convolutional skeleton Transformer (HGCT), to employ the complementary advantages of GCN (i.e., local topology, temporal dynamics and hierarchy) and Transformer (i.e., global context and dynamic attention). HGCT is lightweight and computationally efficient. Quantitative analysis demonstrates the superiority and good interpretability of HGCT.