# Symbiotic Graph Neural Networks for 3D Skeleton-based Human Action Recognition and Motion Prediction

arXiv: https://arxiv.org/abs/1910.02212 · HTML: https://mlsanity.com/paper/1910.02212/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Oct 2019.

## Built on by (most cited first, 30 of 50)

- [Leapfrog Diffusion Model for Stochastic Trajectory Prediction](https://mlsanity.com/paper/2303.10895/) (2023)
- [Graph Foundation Models: Concepts, Opportunities and Challenges](https://mlsanity.com/paper/2310.11829/) (2023)
- [Back to MLP: A Simple Baseline for Human Motion Prediction](https://mlsanity.com/paper/2207.01567/) (2022)
- [EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning](https://mlsanity.com/paper/2303.10876/) (2023)
- [Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2206.13028/) (2022)
- [Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction](https://mlsanity.com/paper/2203.16051/) (2022)
- [Spatio-Temporal Gating-Adjacency GCN for Human Motion Prediction](https://mlsanity.com/paper/2203.01474/) (2022)
- [3Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognition](https://mlsanity.com/paper/2303.14474/) (2023)
- [Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction](https://mlsanity.com/paper/2108.11244/) (2021)
- [Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Space](https://mlsanity.com/paper/2207.07351/) (2022)
- [Conditional Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/2106.11083/) (2021)
- [Dynamic-Group-Aware Networks for Multi-Agent Trajectory Prediction with Relational Reasoning](https://mlsanity.com/paper/2206.13114/) (2022)
- [Spatio-Temporal Tuples Transformer for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2201.02849/) (2022)
- [Joint-Relation Transformer for Multi-Person Motion Prediction](https://mlsanity.com/paper/2308.04808/) (2023)
- [Frequency Guidance Matters: Skeletal Action Recognition by Frequency-Aware Mixed Transformer](https://mlsanity.com/paper/2407.12322/) (2024)
- [Skeleton-based Action Recognition via Temporal-Channel Aggregation](https://mlsanity.com/paper/2205.15936/) (2022)
- [Collaborative Uncertainty Benefits Multi-Agent Multi-Modal Trajectory Forecasting](https://mlsanity.com/paper/2207.05195/) (2022)
- [A Survey on Multimodal Wearable Sensor-based Human Action Recognition](https://mlsanity.com/paper/2404.15349/) (2024)
- [Real-time 3D human action recognition based on Hyperpoint sequence](https://mlsanity.com/paper/2111.08492/) (2021)
- [Dyadic Human Motion Prediction](https://mlsanity.com/paper/2112.00396/) (2021)
- [Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transform](https://mlsanity.com/paper/2505.11892/) (2025)
- [Comparison of Spatio-Temporal Models for Human Motion and Pose Forecasting in Face-to-Face Interaction Scenarios](https://mlsanity.com/paper/2203.03245/) (2022)
- [Best Practices for 2-Body Pose Forecasting](https://mlsanity.com/paper/2304.05758/) (2023)
- [A Decoupled Spatio-Temporal Framework for Skeleton-based Action Segmentation](https://mlsanity.com/paper/2312.05830/) (2023)
- [Ske2Grid: Skeleton-to-Grid Representation Learning for Action Recognition](https://mlsanity.com/paper/2308.07571/) (2023)
- [Only Large Weights (And Not Skip Connections) Can Prevent the Perils of Rank Collapse](https://mlsanity.com/paper/2505.16284/) (2025)
- [Progressive Cross-modal Knowledge Distillation for Human Action Recognition](https://mlsanity.com/paper/2208.08090/) (2022)
- [ClusterViG: Efficient Globally Aware Vision GNNs via Image Partitioning](https://mlsanity.com/paper/2501.10640/) (2025)
- [3D Skeleton-Based Action Recognition: A Review](https://mlsanity.com/paper/2506.00915/) (2025)
- [SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models](https://mlsanity.com/paper/2409.00055/) (2024)

## Abstract (arXiv)

3D skeleton-based action recognition and motion prediction are two essential problems of human activity understanding. In many previous works: 1) they studied two tasks separately, neglecting internal correlations; 2) they did not capture sufficient relations inside the body. To address these issues, we propose a symbiotic model to handle two tasks jointly; and we propose two scales of graphs to explicitly capture relations among body-joints and body-parts. Together, we propose symbiotic graph neural networks, which contain a backbone, an action-recognition head, and a motion-prediction head. Two heads are trained jointly and enhance each other. For the backbone, we propose multi-branch multi-scale graph convolution networks to extract spatial and temporal features. The multi-scale graph convolution networks are based on joint-scale and part-scale graphs. The joint-scale graphs contain actional graphs, capturing action-based relations, and structural graphs, capturing physical constraints. The part-scale graphs integrate body-joints to form specific parts, representing high-level relations. Moreover, dual bone-based graphs and networks are proposed to learn complementary features. We conduct extensive experiments for skeleton-based action recognition and motion prediction with four datasets, NTU-RGB+D, Kinetics, Human3.6M, and CMU Mocap. Experiments show that our symbiotic graph neural networks achieve better performances on both tasks compared to the state-of-the-art methods.