# MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction

arXiv: https://arxiv.org/abs/2108.07152 · HTML: https://mlsanity.com/paper/2108.07152/

## Key facts

- Cited by 100 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 54 references that are in the map.
- First posted on arXiv 16 Aug 2021.

## Builds on (most cited first, 24 of 24)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2003.14111/) (2020)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [Graph Convolutional Networks for Temporal Action Localization](https://mlsanity.com/paper/1909.03252/) (2019)
- [An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1902.09130/) (2019)
- [A Spatio-temporal Transformer for 3D Human Motion Prediction](https://mlsanity.com/paper/2004.08692/) (2020)
- [Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction](https://mlsanity.com/paper/2003.08802/) (2020)
- [An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data](https://mlsanity.com/paper/1611.06067/) (2016)
- [Human Motion Prediction via Spatio-Temporal Inpainting](https://mlsanity.com/paper/1812.05478/) (2018)
- [Graph Convolutional Networks for Hyperspectral Image Classification](https://mlsanity.com/paper/2008.02457/) (2020)
- [BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN](https://mlsanity.com/paper/1812.02591/) (2018)
- [Modeling Human Motion with Quaternion-based Neural Networks](https://mlsanity.com/paper/1901.07677/) (2019)
- [A Hybrid Attention Mechanism for Weakly-Supervised Temporal Action Localization](https://mlsanity.com/paper/2101.00545/) (2021)
- [Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic](https://mlsanity.com/paper/1805.02513/) (2018)
- [Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies](https://mlsanity.com/paper/1902.07367/) (2019)
- [Dual Graph Convolutional Networks with Transformer and Curriculum Learning for Image Captioning](https://mlsanity.com/paper/2108.02366/) (2021)
- [Towards Understanding the Adversarial Vulnerability of Skeleton-based Action Recognition](https://mlsanity.com/paper/2005.07151/) (2020)
- [CRD-CGAN: Category-Consistent and Relativistic Constraints for Diverse Text-to-Image Generation](https://mlsanity.com/paper/2107.13516/) (2021)
- [TrajectoryNet: a new spatio-temporal feature learning network for human motion prediction](https://mlsanity.com/paper/1910.06583/) (2019)
- [Early Action Prediction with Generative Adversarial Networks](https://mlsanity.com/paper/1904.13085/) (2019)

## Built on by (most cited first, 30 of 100)

- [Back to MLP: A Simple Baseline for Human Motion Prediction](https://mlsanity.com/paper/2207.01567/) (2022)
- [EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning](https://mlsanity.com/paper/2303.10876/) (2023)
- [BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction](https://mlsanity.com/paper/2211.14304/) (2022)
- [Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction](https://mlsanity.com/paper/2203.16051/) (2022)
- [The Wanderings of Odysseus in 3D Scenes](https://mlsanity.com/paper/2112.09251/) (2021)
- [MotionMixer: MLP-based 3D Human Body Pose Forecasting](https://mlsanity.com/paper/2207.00499/) (2022)
- [Generating Continual Human Motion in Diverse 3D Scenes](https://mlsanity.com/paper/2304.02061/) (2023)
- [Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchors](https://mlsanity.com/paper/2302.04860/) (2023)
- [Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Space](https://mlsanity.com/paper/2207.07351/) (2022)
- [Skeleton-Parted Graph Scattering Networks for 3D Human Motion Prediction](https://mlsanity.com/paper/2208.00368/) (2022)
- [Rethinking Learning Approaches for Long-Term Action Anticipation](https://mlsanity.com/paper/2210.11566/) (2022)
- [Human Joint Kinematics Diffusion-Refinement for Stochastic Motion Prediction](https://mlsanity.com/paper/2210.05976/) (2022)
- [SocialCircle: Learning the Angle-based Social Interaction Representation for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2310.05370/) (2023)
- [Trajectory-Aware Body Interaction Transformer for Multi-Person Pose Forecasting](https://mlsanity.com/paper/2303.05095/) (2023)
- [Plan, Posture and Go: Towards Open-World Text-to-Motion Generation](https://mlsanity.com/paper/2312.14828/) (2023)
- [Pose Forecasting in Industrial Human-Robot Collaboration](https://mlsanity.com/paper/2208.07308/) (2022)
- [CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion](https://mlsanity.com/paper/2305.12554/) (2023)
- [Auxiliary Tasks Benefit 3D Skeleton-based Human Motion Prediction](https://mlsanity.com/paper/2308.08942/) (2023)
- [TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Prediction](https://mlsanity.com/paper/2307.16106/) (2023)
- [HOIMotion: Forecasting Human Motion During Human-Object Interactions Using Egocentric 3D Object Bounding Boxes](https://mlsanity.com/paper/2407.02633/) (2024)
- [UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generation](https://mlsanity.com/paper/2508.01126/) (2025)
- [SkinningNet: Two-Stream Graph Convolutional Neural Network for Skinning Prediction of Synthetic Characters](https://mlsanity.com/paper/2203.04746/) (2022)
- [HiT-DVAE: Human Motion Generation via Hierarchical Transformer Dynamical VAE](https://mlsanity.com/paper/2204.01565/) (2022)
- [Social Interpretable Tree for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2205.13296/) (2022)
- [Another Vertical View: A Hierarchical Network for Heterogeneous Trajectory Prediction via Spectrums](https://mlsanity.com/paper/2304.05106/) (2023)
- [Learning Semantic Latent Directions for Accurate and Controllable Human Motion Prediction](https://mlsanity.com/paper/2407.11494/) (2024)
- [Continuous Intermediate Token Learning with Implicit Motion Manifold for Keyframe Based Motion Interpolation](https://mlsanity.com/paper/2303.14926/) (2023)
- [GCNext: Towards the Unity of Graph Convolutions for Human Motion Prediction](https://mlsanity.com/paper/2312.11850/) (2023)
- [Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Prediction](https://mlsanity.com/paper/2501.06035/) (2025)
- [Toward Reliable Human Pose Forecasting with Uncertainty](https://mlsanity.com/paper/2304.06707/) (2023)

## Abstract (arXiv)

Human motion prediction is a challenging task due to the stochasticity and aperiodicity of future poses. Recently, graph convolutional network has been proven to be very effective to learn dynamic relations among pose joints, which is helpful for pose prediction. On the other hand, one can abstract a human pose recursively to obtain a set of poses at multiple scales. With the increase of the abstraction level, the motion of the pose becomes more stable, which benefits pose prediction too. In this paper, we propose a novel Multi-Scale Residual Graph Convolution Network (MSR-GCN) for human pose prediction task in the manner of end-to-end. The GCNs are used to extract features from fine to coarse scale and then from coarse to fine scale. The extracted features at each scale are then combined and decoded to obtain the residuals between the input and target poses. Intermediate supervisions are imposed on all the predicted poses, which enforces the network to learn more representative features. Our proposed approach is evaluated on two standard benchmark datasets, i.e., the Human3.6M dataset and the CMU Mocap dataset. Experimental results demonstrate that our method outperforms the state-of-the-art approaches. Code and pre-trained models are available at https://github.com/Droliven/MSRGCN.