# Multi-level Motion Attention for Human Motion Prediction

arXiv: https://arxiv.org/abs/2106.09300 · HTML: https://mlsanity.com/paper/2106.09300/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [History Repeats Itself: Human Motion Prediction via Motion Attention](https://mlsanity.com/paper/2007.11755/) (2020)
- [Convolutional Sequence to Sequence Model for Human Dynamics](https://mlsanity.com/paper/1805.00655/) (2018)
- [Human Motion Prediction via Spatio-Temporal Inpainting](https://mlsanity.com/paper/1812.05478/) (2018)
- [Deep representation learning for human motion prediction and classification](https://mlsanity.com/paper/1702.07486/) (2017)
- [A Neural Temporal Model for Human Motion Prediction](https://mlsanity.com/paper/1809.03036/) (2018)
- [Modeling Human Motion with Quaternion-based Neural Networks](https://mlsanity.com/paper/1901.07677/) (2019)
- [Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic](https://mlsanity.com/paper/1805.02513/) (2018)
- [Real-World Repetition Estimation by Div, Grad and Curl](https://mlsanity.com/paper/1802.09971/) (2018)
- [Structure from Recurrent Motion: From Rigidity to Recurrency](https://mlsanity.com/paper/1804.06510/) (2018)

## Built on by (most cited first, 30 of 35)

- [Generating Smooth Pose Sequences for Diverse Human Motion Prediction](https://mlsanity.com/paper/2108.08422/) (2021)
- [Spatio-Temporal Gating-Adjacency GCN for Human Motion Prediction](https://mlsanity.com/paper/2203.01474/) (2022)
- [Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Models](https://mlsanity.com/paper/2310.02242/) (2023)
- [MotionMixer: MLP-based 3D Human Body Pose Forecasting](https://mlsanity.com/paper/2207.00499/) (2022)
- [Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Space](https://mlsanity.com/paper/2207.07351/) (2022)
- [3D Human Motion Prediction: A Survey](https://mlsanity.com/paper/2203.01593/) (2022)
- [SoMoFormer: Multi-Person Pose Forecasting with Transformers](https://mlsanity.com/paper/2208.14023/) (2022)
- [CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion](https://mlsanity.com/paper/2305.12554/) (2023)
- [Language-Driven Interactive Traffic Trajectory Generation](https://mlsanity.com/paper/2405.15388/) (2024)
- [HOIMotion: Forecasting Human Motion During Human-Object Interactions Using Egocentric 3D Object Bounding Boxes](https://mlsanity.com/paper/2407.02633/) (2024)
- [Learning Semantic Latent Directions for Accurate and Controllable Human Motion Prediction](https://mlsanity.com/paper/2407.11494/) (2024)
- [Didn't see that coming: a survey on non-verbal social human behavior forecasting](https://mlsanity.com/paper/2203.02480/) (2022)
- [GCNext: Towards the Unity of Graph Convolutions for Human Motion Prediction](https://mlsanity.com/paper/2312.11850/) (2023)
- [Dyadic Human Motion Prediction](https://mlsanity.com/paper/2112.00396/) (2021)
- [Comparison of Spatio-Temporal Models for Human Motion and Pose Forecasting in Face-to-Face Interaction Scenarios](https://mlsanity.com/paper/2203.03245/) (2022)
- [Best Practices for 2-Body Pose Forecasting](https://mlsanity.com/paper/2304.05758/) (2023)
- [Staged Contact-Aware Global Human Motion Forecasting](https://mlsanity.com/paper/2309.08947/) (2023)
- [Context-based Interpretable Spatio-Temporal Graph Convolutional Network for Human Motion Forecasting](https://mlsanity.com/paper/2402.19237/) (2024)
- [Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Prediction](https://mlsanity.com/paper/2405.18700/) (2024)
- [Learning Disentangled Representations for Controllable Human Motion Prediction](https://mlsanity.com/paper/2207.01388/) (2022)
- [GazeMotion: Gaze-guided Human Motion Forecasting](https://mlsanity.com/paper/2403.09885/) (2024)
- [Towards Accurate Human Motion Prediction via Iterative Refinement](https://mlsanity.com/paper/2305.04443/) (2023)
- [Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Prediction](https://mlsanity.com/paper/2310.14907/) (2023)
- [Learning to Predict Diverse Human Motions from a Single Image via Mixture Density Networks](https://mlsanity.com/paper/2109.05776/) (2021)
- [Dynamic Compositional Graph Convolutional Network for Efficient Composite Human Motion Prediction](https://mlsanity.com/paper/2311.13781/) (2023)
- [Few-shot human motion prediction for heterogeneous sensors](https://mlsanity.com/paper/2212.11771/) (2022)
- [Physics-constrained Attack against Convolution-based Human Motion Prediction](https://mlsanity.com/paper/2306.11990/) (2023)
- [Graph-Guided MLP-Mixer for Skeleton-Based Human Motion Prediction](https://mlsanity.com/paper/2304.03532/) (2023)
- [Recent Advances in Deterministic Human Motion Prediction: A Review](https://mlsanity.com/paper/2312.06184/) (2023)
- [Fooling Neural Networks for Motion Forecasting via Adversarial Attacks](https://mlsanity.com/paper/2403.04954/) (2024)

## Abstract (arXiv)

Human motion prediction aims to forecast future human poses given a historical motion. Whether based on recurrent or feed-forward neural networks, existing learning based methods fail to model the observation that human motion tends to repeat itself, even for complex sports actions and cooking activities. Here, we introduce an attention based feed-forward network that explicitly leverages this observation. In particular, instead of modeling frame-wise attention via pose similarity, we propose to extract motion attention to capture the similarity between the current motion context and the historical motion sub-sequences. In this context, we study the use of different types of attention, computed at joint, body part, and full pose levels. Aggregating the relevant past motions and processing the result with a graph convolutional network allows us to effectively exploit motion patterns from the long-term history to predict the future poses. Our experiments on Human3.6M, AMASS and 3DPW validate the benefits of our approach for both periodical and non-periodical actions. Thanks to our attention model, it yields state-of-the-art results on all three datasets. Our code is available at https://github.com/wei-mao-2019/HisRepItself.