# Aggregated Multi-GANs for Controlled 3D Human Motion Prediction

arXiv: https://arxiv.org/abs/2103.09755 · HTML: https://mlsanity.com/paper/2103.09755/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 17 Mar 2021.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [HP-GAN: Probabilistic 3D human motion prediction via GAN](https://mlsanity.com/paper/1711.09561/) (2017)
- [Convolutional Sequence to Sequence Model for Human Dynamics](https://mlsanity.com/paper/1805.00655/) (2018)
- [Deep representation learning for human motion prediction and classification](https://mlsanity.com/paper/1702.07486/) (2017)
- [MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics](https://mlsanity.com/paper/1808.04545/) (2018)
- [Diverse Trajectory Forecasting with Determinantal Point Processes](https://mlsanity.com/paper/1907.04967/) (2019)
- [BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN](https://mlsanity.com/paper/1812.02591/) (2018)
- [Modeling Human Motion with Quaternion-based Neural Networks](https://mlsanity.com/paper/1901.07677/) (2019)
- [Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies](https://mlsanity.com/paper/1902.07367/) (2019)
- [Accurate Temporal Action Proposal Generation with Relation-Aware Pyramid Network](https://mlsanity.com/paper/2003.04145/) (2020)
- [Direct Prediction of 3D Body Poses from Motion Compensated Sequences](https://mlsanity.com/paper/1511.06692/) (2015)
- [Unsupervised Online Video Object Segmentation with Motion Property Understanding](https://mlsanity.com/paper/1810.03783/) (2018)

## Built on by (most cited first, 23 of 23)

- [IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions](https://mlsanity.com/paper/2212.07555/) (2022)
- [BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction](https://mlsanity.com/paper/2211.14304/) (2022)
- [Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Space](https://mlsanity.com/paper/2207.07351/) (2022)
- [3D Human Motion Prediction: A Survey](https://mlsanity.com/paper/2203.01593/) (2022)
- [Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2203.15227/) (2022)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [Motion Prediction via Joint Dependency Modeling in Phase Space](https://mlsanity.com/paper/2201.02365/) (2022)
- [Copy Motion From One to Another: Fake Motion Video Generation](https://mlsanity.com/paper/2205.01373/) (2022)
- [Learning Human Motion Prediction via Stochastic Differential Equations](https://mlsanity.com/paper/2112.11124/) (2021)
- [CgT-GAN: CLIP-guided Text GAN for Image Captioning](https://mlsanity.com/paper/2308.12045/) (2023)
- [Do As I Do: Pose Guided Human Motion Copy](https://mlsanity.com/paper/2406.16601/) (2024)
- [Learning Constrained Dynamic Correlations in Spatiotemporal Graphs for Motion Prediction](https://mlsanity.com/paper/2204.01297/) (2022)
- [Action Recognition with Multi-stream Motion Modeling and Mutual Information Maximization](https://mlsanity.com/paper/2306.07576/) (2023)
- [EgoAgent: A Joint Predictive Agent Model in Egocentric Worlds](https://mlsanity.com/paper/2502.05857/) (2025)
- [Multimodal Sense-Informed Prediction of 3D Human Motions](https://mlsanity.com/paper/2405.02911/) (2024)
- [Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Prediction](https://mlsanity.com/paper/2405.18700/) (2024)
- [FreeMotion: MoCap-Free Human Motion Synthesis with Multimodal Large Language Models](https://mlsanity.com/paper/2406.10740/) (2024)
- [Learning Disentangled Representations for Controllable Human Motion Prediction](https://mlsanity.com/paper/2207.01388/) (2022)
- [Multi-Scale Control Signal-Aware Transformer for Motion Synthesis without Phase](https://mlsanity.com/paper/2303.01685/) (2023)
- [HVIS: A Human-like Vision and Inference System for Human Motion Prediction](https://mlsanity.com/paper/2502.16913/) (2025)
- [Meta-Auxiliary Learning for Adaptive Human Pose Prediction](https://mlsanity.com/paper/2304.06411/) (2023)
- [Multimodal Generative AI with Autoregressive LLMs for Human Motion Understanding and Generation: A Way Forward](https://mlsanity.com/paper/2506.03191/) (2025)
- [Diffusion Model-based Activity Completion for AI Motion Capture from Videos](https://mlsanity.com/paper/2505.21566/) (2025)

## Abstract (arXiv)

Human motion prediction from historical pose sequence is at the core of many applications in machine intelligence. However, in current state-of-the-art methods, the predicted future motion is confined within the same activity. One can neither generate predictions that differ from the current activity, nor manipulate the body parts to explore various future possibilities. Undoubtedly, this greatly limits the usefulness and applicability of motion prediction. In this paper, we propose a generalization of the human motion prediction task in which control parameters can be readily incorporated to adjust the forecasted motion. Our method is compelling in that it enables manipulable motion prediction across activity types and allows customization of the human movement in a variety of fine-grained ways. To this aim, a simple yet effective composite GAN structure, consisting of local GANs for different body parts and aggregated via a global GAN is presented. The local GANs game in lower dimensions, while the global GAN adjusts in high dimensional space to avoid mode collapse. Extensive experiments show that our method outperforms state-of-the-art. The codes are available at https://github.com/herolvkd/AM-GAN.