# Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies

arXiv: https://arxiv.org/abs/1902.07367 · HTML: https://mlsanity.com/paper/1902.07367/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2019.

## Built on by (most cited first, 30 of 34)

- [FLAME: Free-form Language-based Motion Synthesis & Editing](https://mlsanity.com/paper/2209.00349/) (2022)
- [Hierarchical Fine-Grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2303.17111/) (2023)
- [MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction](https://mlsanity.com/paper/2108.07152/) (2021)
- [EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning](https://mlsanity.com/paper/2303.10876/) (2023)
- [Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction](https://mlsanity.com/paper/2203.16051/) (2022)
- [Multi-domain Learning for Updating Face Anti-spoofing Models](https://mlsanity.com/paper/2208.11148/) (2022)
- [Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction](https://mlsanity.com/paper/2108.11244/) (2021)
- [Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction](https://mlsanity.com/paper/2112.15012/) (2021)
- [3D Human Motion Prediction: A Survey](https://mlsanity.com/paper/2203.01593/) (2022)
- [Skeleton-Parted Graph Scattering Networks for 3D Human Motion Prediction](https://mlsanity.com/paper/2208.00368/) (2022)
- [Aggregated Multi-GANs for Controlled 3D Human Motion Prediction](https://mlsanity.com/paper/2103.09755/) (2021)
- [Auxiliary Tasks Benefit 3D Skeleton-based Human Motion Prediction](https://mlsanity.com/paper/2308.08942/) (2023)
- [Motion Prediction via Joint Dependency Modeling in Phase Space](https://mlsanity.com/paper/2201.02365/) (2022)
- [MUGL: Large Scale Multi Person Conditional Action Generation with Locomotion](https://mlsanity.com/paper/2110.11460/) (2021)
- [Continuous Intermediate Token Learning with Implicit Motion Manifold for Keyframe Based Motion Interpolation](https://mlsanity.com/paper/2303.14926/) (2023)
- [Pedestrian Crossing Action Recognition and Trajectory Prediction with 3D Human Keypoints](https://mlsanity.com/paper/2306.01075/) (2023)
- [Dynamic Dense Graph Convolutional Network for Skeleton-based Human Motion Prediction](https://mlsanity.com/paper/2311.17408/) (2023)
- [Conditional Temporal Variational AutoEncoder for Action Video Prediction](https://mlsanity.com/paper/2108.05658/) (2021)
- [Human Part-wise 3D Motion Context Learning for Sign Language Recognition](https://mlsanity.com/paper/2308.09305/) (2023)
- [Tracing Hyperparameter Dependencies for Model Parsing via Learnable Graph Pooling Network](https://mlsanity.com/paper/2312.02224/) (2023)
- [Learning Constrained Dynamic Correlations in Spatiotemporal Graphs for Motion Prediction](https://mlsanity.com/paper/2204.01297/) (2022)
- [Context-based Interpretable Spatio-Temporal Graph Convolutional Network for Human Motion Forecasting](https://mlsanity.com/paper/2402.19237/) (2024)
- [Multimodal Sense-Informed Prediction of 3D Human Motions](https://mlsanity.com/paper/2405.02911/) (2024)
- [Towards more realistic human motion prediction with attention to motion coordination](https://mlsanity.com/paper/2404.03584/) (2024)
- [Multiscale Residual Learning of Graph Convolutional Sequence Chunks for Human Motion Prediction](https://mlsanity.com/paper/2308.16801/) (2023)
- [HVIS: A Human-like Vision and Inference System for Human Motion Prediction](https://mlsanity.com/paper/2502.16913/) (2025)
- [Non-local Graph Convolutional Network for joint Activity Recognition and Motion Prediction](https://mlsanity.com/paper/2108.01518/) (2021)
- [Neural Marionette: Unsupervised Learning of Motion Skeleton and Latent Dynamics from Volumetric Video](https://mlsanity.com/paper/2202.08418/) (2022)
- [Remembering What Is Important: A Factorised Multi-Head Retrieval and Auxiliary Memory Stabilisation Scheme for Human Motion Prediction](https://mlsanity.com/paper/2305.11394/) (2023)
- [Physics-constrained Attack against Convolution-based Human Motion Prediction](https://mlsanity.com/paper/2306.11990/) (2023)

## Abstract (arXiv)

Human motion prediction from motion capture data is a classical problem in the computer vision, and conventional methods take the holistic human body as input. These methods ignore the fact that, in various human activities, different body components (limbs and the torso) have distinctive characteristics in terms of the moving pattern. In this paper, we argue local representations on different body components should be learned separately and, based on such idea, propose a network, Skeleton Network (SkelNet), for long-term human motion prediction. Specifically, at each time-step, local structure representations of input (human body) are obtained via SkelNet's branches of component-specific layers, then the shared layer uses local spatial representations to predict the future human pose. Our SkelNet is the first to use local structure representations for predicting the human motion. Then, for short-term human motion prediction, we propose the second network, named as Skeleton Temporal Network (Skel-TNet). Skel-TNet consists of three components: SkelNet and a Recurrent Neural Network, they have advantages in learning spatial and temporal dependencies for predicting human motion, respectively; a feed-forward network that outputs the final estimation. Our methods achieve promising results on the Human3.6M dataset and the CMU motion capture dataset.