# Learning Multiscale Correlations for Human Motion Prediction

arXiv: https://arxiv.org/abs/2103.10674 · HTML: https://mlsanity.com/paper/2103.10674/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 19 Mar 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles](https://mlsanity.com/paper/1604.07446/) (2016)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [History Repeats Itself: Human Motion Prediction via Motion Attention](https://mlsanity.com/paper/2007.11755/) (2020)
- [Scale-Aware Trident Networks for Object Detection](https://mlsanity.com/paper/1901.01892/) (2019)
- [Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction](https://mlsanity.com/paper/2003.08802/) (2020)
- [Convolutional Sequence to Sequence Model for Human Dynamics](https://mlsanity.com/paper/1805.00655/) (2018)
- [QuaterNet: A Quaternion-based Recurrent Model for Human Motion](https://mlsanity.com/paper/1805.06485/) (2018)
- [Deep representation learning for human motion prediction and classification](https://mlsanity.com/paper/1702.07486/) (2017)
- [Imitation Learning for Human Pose Prediction](https://mlsanity.com/paper/1909.03449/) (2019)
- [Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic](https://mlsanity.com/paper/1805.02513/) (2018)
- [Motion Prediction Using Temporal Inception Module](https://mlsanity.com/paper/2010.03006/) (2020)
- [Complex Sequential Understanding through the Awareness of Spatial and Temporal Concepts](https://mlsanity.com/paper/2006.00212/) (2020)
- [Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network](https://mlsanity.com/paper/1911.02404/) (2019)

## Built on by (most cited first, 9 of 9)

- [3D Human Motion Prediction: A Survey](https://mlsanity.com/paper/2203.01593/) (2022)
- [Dynamic Dense Graph Convolutional Network for Skeleton-based Human Motion Prediction](https://mlsanity.com/paper/2311.17408/) (2023)
- [Context-based Interpretable Spatio-Temporal Graph Convolutional Network for Human Motion Forecasting](https://mlsanity.com/paper/2402.19237/) (2024)
- [HVIS: A Human-like Vision and Inference System for Human Motion Prediction](https://mlsanity.com/paper/2502.16913/) (2025)
- [Remembering What Is Important: A Factorised Multi-Head Retrieval and Auxiliary Memory Stabilisation Scheme for Human Motion Prediction](https://mlsanity.com/paper/2305.11394/) (2023)
- [Graph-Guided MLP-Mixer for Skeleton-Based Human Motion Prediction](https://mlsanity.com/paper/2304.03532/) (2023)
- [Learning Snippet-to-Motion Progression for Skeleton-based Human Motion Prediction](https://mlsanity.com/paper/2307.14006/) (2023)
- [Multimodal Generative AI with Autoregressive LLMs for Human Motion Understanding and Generation: A Way Forward](https://mlsanity.com/paper/2506.03191/) (2025)
- [Personalized Pose Forecasting](https://mlsanity.com/paper/2312.03528/) (2023)

## Abstract (arXiv)

In spite of the great progress in human motion prediction, it is still a challenging task to predict those aperiodic and complicated motions. We believe that to capture the correlations among human body components is the key to understand the human motion. In this paper, we propose a novel multiscale graph convolution network (MGCN) to address this problem. Firstly, we design an adaptive multiscale interactional encoding module (MIEM) which is composed of two sub modules: scale transformation module and scale interaction module to learn the human body correlations. Secondly, we apply a coarse-to-fine decoding strategy to decode the motions sequentially. We evaluate our approach on two standard benchmark datasets for human motion prediction: Human3.6M and CMU motion capture dataset. The experiments show that the proposed approach achieves the state-of-the-art performance for both short-term and long-term prediction especially in those complicated action category.