# Imitation Learning for Human Pose Prediction

arXiv: https://arxiv.org/abs/1909.03449 · HTML: https://mlsanity.com/paper/1909.03449/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 8 Sep 2019.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 29 of 29)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [Generating Videos with Scene Dynamics](https://mlsanity.com/paper/1609.02612/) (2016)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [HP-GAN: Probabilistic 3D human motion prediction via GAN](https://mlsanity.com/paper/1711.09561/) (2017)
- [VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera](https://mlsanity.com/paper/1705.01583/) (2017)
- [Peeking into the Future: Predicting Future Person Activities and Locations in Videos](https://mlsanity.com/paper/1902.03748/) (2019)
- [Procedure Planning in Instructional Videos](https://mlsanity.com/paper/1907.01172/) (2019)
- [The Pose Knows: Video Forecasting by Generating Pose Futures](https://mlsanity.com/paper/1705.00053/) (2017)
- [An Uncertain Future: Forecasting from Static Images using Variational Autoencoders](https://mlsanity.com/paper/1606.07873/) (2016)
- [MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics](https://mlsanity.com/paper/1808.04545/) (2018)
- [Learning Human Motion Models for Long-term Predictions](https://mlsanity.com/paper/1704.02827/) (2017)
- [Learning Plannable Representations with Causal InfoGAN](https://mlsanity.com/paper/1807.09341/) (2018)
- [Action-Agnostic Human Pose Forecasting](https://mlsanity.com/paper/1810.09676/) (2018)
- [Socially Compliant Navigation through Raw Depth Inputs with Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1710.02543/) (2017)
- [Predicting Deeper into the Future of Semantic Segmentation](https://mlsanity.com/paper/1703.07684/) (2017)
- [Dense Optical Flow Prediction from a Static Image](https://mlsanity.com/paper/1505.00295/) (2015)
- [Wasserstein Divergence for GANs](https://mlsanity.com/paper/1712.01026/) (2017)
- [Forecasting Human Dynamics from Static Images](https://mlsanity.com/paper/1704.03432/) (2017)
- [Visual Forecasting by Imitating Dynamics in Natural Sequences](https://mlsanity.com/paper/1708.05827/) (2017)
- [Geometry-Based Next Frame Prediction from Monocular Video](https://mlsanity.com/paper/1609.06377/) (2016)

## Built on by (most cited first, 30 of 44)

- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022)
- [SAGA: Stochastic Whole-Body Grasping with Contact](https://mlsanity.com/paper/2112.10103/) (2021)
- [Space-Time-Separable Graph Convolutional Network for Pose Forecasting](https://mlsanity.com/paper/2110.04573/) (2021)
- [Generating Smooth Pose Sequences for Diverse Human Motion Prediction](https://mlsanity.com/paper/2108.08422/) (2021)
- [TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild](https://mlsanity.com/paper/2104.04029/) (2021)
- [Spatio-Temporal Gating-Adjacency GCN for Human Motion Prediction](https://mlsanity.com/paper/2203.01474/) (2022)
- [Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchors](https://mlsanity.com/paper/2302.04860/) (2023)
- [The Boltzmann Policy Distribution: Accounting for Systematic Suboptimality in Human Models](https://mlsanity.com/paper/2204.10759/) (2022)
- [Conditional Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/2106.11083/) (2021)
- [Contact-aware Human Motion Forecasting](https://mlsanity.com/paper/2210.03954/) (2022)
- [Weakly-supervised Action Transition Learning for Stochastic Human Motion Prediction](https://mlsanity.com/paper/2205.15608/) (2022)
- [MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control](https://mlsanity.com/paper/2208.07363/) (2022)
- [UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generation](https://mlsanity.com/paper/2508.01126/) (2025)
- [MotionAug: Augmentation with Physical Correction for Human Motion Prediction](https://mlsanity.com/paper/2203.09116/) (2022)
- [3D Pose Based Feedback for Physical Exercises](https://mlsanity.com/paper/2208.03257/) (2022)
- [MaIL: Improving Imitation Learning with Mamba](https://mlsanity.com/paper/2406.08234/) (2024)
- [Learning Multiscale Correlations for Human Motion Prediction](https://mlsanity.com/paper/2103.10674/) (2021)
- [Conditional Temporal Variational AutoEncoder for Action Video Prediction](https://mlsanity.com/paper/2108.05658/) (2021)
- [Dyadic Human Motion Prediction](https://mlsanity.com/paper/2112.00396/) (2021)
- [Social Motion Prediction with Cognitive Hierarchies](https://mlsanity.com/paper/2311.04726/) (2023)
- [Best Practices for 2-Body Pose Forecasting](https://mlsanity.com/paper/2304.05758/) (2023)
- [Scene-aware Human Motion Forecasting via Mutual Distance Prediction](https://mlsanity.com/paper/2310.00615/) (2023)
- [On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstration](https://mlsanity.com/paper/2010.08353/) (2020)
- [SoMoFormer: Social-Aware Motion Transformer for Multi-Person Motion Prediction](https://mlsanity.com/paper/2208.09224/) (2022)
- [Review of Video Predictive Understanding: Early Action Recognition and Future Action Prediction](https://mlsanity.com/paper/2107.05140/) (2021)
- [FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations](https://mlsanity.com/paper/2211.14309/) (2022)
- [RoboSSM: Scalable In-context Imitation Learning via State-Space Models](https://mlsanity.com/paper/2509.19658/) (2025)
- [Uncertainty-aware Human Motion Prediction](https://mlsanity.com/paper/2107.03575/) (2021)
- [SPOTR: Spatio-temporal Pose Transformers for Human Motion Prediction](https://mlsanity.com/paper/2303.06277/) (2023)

## Abstract (arXiv)

Modeling and prediction of human motion dynamics has long been a challenging problem in computer vision, and most existing methods rely on the end-to-end supervised training of various architectures of recurrent neural networks. Inspired by the recent success of deep reinforcement learning methods, in this paper we propose a new reinforcement learning formulation for the problem of human pose prediction, and develop an imitation learning algorithm for predicting future poses under this formulation through a combination of behavioral cloning and generative adversarial imitation learning. Our experiments show that our proposed method outperforms all existing state-of-the-art baseline models by large margins on the task of human pose prediction in both short-term predictions and long-term predictions, while also enjoying huge advantage in training speed.