# Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Space

arXiv: https://arxiv.org/abs/2207.07351 · HTML: https://mlsanity.com/paper/2207.07351/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 53 references that are in the map.
- First posted on arXiv 15 Jul 2022.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)
- [HP-GAN: Probabilistic 3D human motion prediction via GAN](https://mlsanity.com/paper/1711.09561/) (2017)
- [History Repeats Itself: Human Motion Prediction via Motion Attention](https://mlsanity.com/paper/2007.11755/) (2020)
- [A Spatio-temporal Transformer for 3D Human Motion Prediction](https://mlsanity.com/paper/2004.08692/) (2020)
- [Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction](https://mlsanity.com/paper/2003.08802/) (2020)
- [A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Prediction](https://mlsanity.com/paper/2108.06852/) (2021)
- [An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data](https://mlsanity.com/paper/1611.06067/) (2016)
- [MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction](https://mlsanity.com/paper/2108.07152/) (2021)
- [Structured Prediction Helps 3D Human Motion Modelling](https://mlsanity.com/paper/1910.09070/) (2019)
- [The Pose Knows: Video Forecasting by Generating Pose Futures](https://mlsanity.com/paper/1705.00053/) (2017)
- [MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics](https://mlsanity.com/paper/1808.04545/) (2018)
- [Context-aware Human Motion Prediction](https://mlsanity.com/paper/1904.03419/) (2019)
- [Diverse Trajectory Forecasting with Determinantal Point Processes](https://mlsanity.com/paper/1907.04967/) (2019)
- [DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data](https://mlsanity.com/paper/1706.02071/) (2017)
- [Generating Smooth Pose Sequences for Diverse Human Motion Prediction](https://mlsanity.com/paper/2108.08422/) (2021)
- [Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction](https://mlsanity.com/paper/2203.16051/) (2022)
- [Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective](https://mlsanity.com/paper/1806.07772/) (2018)
- [Action-Agnostic Human Pose Forecasting](https://mlsanity.com/paper/1810.09676/) (2018)
- [BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN](https://mlsanity.com/paper/1812.02591/) (2018)
- [Symbiotic Graph Neural Networks for 3D Skeleton-based Human Action Recognition and Motion Prediction](https://mlsanity.com/paper/1910.02212/) (2019)
- [Pose Transformers (POTR): Human Motion Prediction with Non-Autoregressive Transformers](https://mlsanity.com/paper/2109.07531/) (2021)
- [Multi-level Motion Attention for Human Motion Prediction](https://mlsanity.com/paper/2106.09300/) (2021)
- [Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction](https://mlsanity.com/paper/2112.15012/) (2021)
- [Dual Graph Convolutional Networks with Transformer and Curriculum Learning for Image Captioning](https://mlsanity.com/paper/2108.02366/) (2021)
- [Aggregated Multi-GANs for Controlled 3D Human Motion Prediction](https://mlsanity.com/paper/2103.09755/) (2021)
- [Multitask Non-Autoregressive Model for Human Motion Prediction](https://mlsanity.com/paper/2007.06426/) (2020)

## Built on by (most cited first, 30 of 32)

- [BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction](https://mlsanity.com/paper/2211.14304/) (2022)
- [HumanMAC: Masked Motion Completion for Human Motion Prediction](https://mlsanity.com/paper/2302.03665/) (2023)
- [Interactive Humanoid: Online Full-Body Motion Reaction Synthesis with Social Affordance Canonicalization and Forecasting](https://mlsanity.com/paper/2312.08983/) (2023)
- [CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion](https://mlsanity.com/paper/2305.12554/) (2023)
- [TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Prediction](https://mlsanity.com/paper/2307.16106/) (2023)
- [Learning Semantic Latent Directions for Accurate and Controllable Human Motion Prediction](https://mlsanity.com/paper/2407.11494/) (2024)
- [Continuous Intermediate Token Learning with Implicit Motion Manifold for Keyframe Based Motion Interpolation](https://mlsanity.com/paper/2303.14926/) (2023)
- [Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Prediction](https://mlsanity.com/paper/2501.06035/) (2025)
- [SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios](https://mlsanity.com/paper/2506.02444/) (2025)
- [MotionWavelet: Human Motion Prediction via Wavelet Manifold Learning](https://mlsanity.com/paper/2411.16964/) (2024)
- [MotionMap: Representing Multimodality in Human Pose Forecasting](https://mlsanity.com/paper/2412.18883/) (2024)
- [Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Prediction](https://mlsanity.com/paper/2405.18700/) (2024)
- [DivDiff: A Conditional Diffusion Model for Diverse Human Motion Prediction](https://mlsanity.com/paper/2409.00014/) (2024)
- [FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations](https://mlsanity.com/paper/2211.14309/) (2022)
- [InterAgent: Physics-based Multi-agent Command Execution via Diffusion on Interaction Graphs](https://mlsanity.com/paper/2512.07410/) (2025)
- [Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Prediction](https://mlsanity.com/paper/2310.14907/) (2023)
- [CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flow](https://mlsanity.com/paper/2505.13140/) (2025)
- [mmPred: Radar-based Human Motion Prediction in the Dark](https://mlsanity.com/paper/2512.00345/) (2025)
- [Towards Practical Human Motion Prediction with LiDAR Point Clouds](https://mlsanity.com/paper/2408.08202/) (2024)
- [Uncertainty-aware Probabilistic 3D Human Motion Forecasting via Invertible Networks](https://mlsanity.com/paper/2507.14694/) (2025)
- [A Spatio-temporal Continuous Network for Stochastic 3D Human Motion Prediction](https://mlsanity.com/paper/2508.01585/) (2025)
- [Incorporating Test-Time Optimization into Training with Dual Networks for Human Mesh Recovery](https://mlsanity.com/paper/2401.14121/) (2024)
- [HumanCM: One Step Human Motion Prediction](https://mlsanity.com/paper/2510.16709/) (2025)
- [MMCM: Multimodality-aware Metric using Clustering-based Modes for Probabilistic Human Motion Prediction](https://mlsanity.com/paper/2511.15179/) (2025)
- [Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic](https://mlsanity.com/paper/2507.04062/) (2025)
- [SMamDiff: Spatial Mamba for Stochastic Human Motion Prediction](https://mlsanity.com/paper/2512.00355/) (2025)
- [SimpliHuMoN: Simplifying Human Motion Prediction](https://mlsanity.com/paper/2603.04399/) (2026)
- [KHMP: Frequency-Domain Kalman Refinement for High-Fidelity Human Motion Prediction](https://mlsanity.com/paper/2603.21327/) (2026)
- [Capturing Uncertainty in Human Motion for Representation Learning in Soccer](https://mlsanity.com/paper/2608.11203/) (2026)
- [Gaussian-Mixture Latent Flow for Stochastic 3D Human Motion Prediction](https://mlsanity.com/paper/2608.21093/) (2026)

## Abstract (arXiv)

Diverse human motion prediction aims at predicting multiple possible future pose sequences from a sequence of observed poses. Previous approaches usually employ deep generative networks to model the conditional distribution of data, and then randomly sample outcomes from the distribution. While different results can be obtained, they are usually the most likely ones which are not diverse enough. Recent work explicitly learns multiple modes of the conditional distribution via a deterministic network, which however can only cover a fixed number of modes within a limited range. In this paper, we propose a novel sampling strategy for sampling very diverse results from an imbalanced multimodal distribution learned by a deep generative model. Our method works by generating an auxiliary space and smartly making randomly sampling from the auxiliary space equivalent to the diverse sampling from the target distribution. We propose a simple yet effective network architecture that implements this novel sampling strategy, which incorporates a Gumbel-Softmax coefficient matrix sampling method and an aggressive diversity promoting hinge loss function. Extensive experiments demonstrate that our method significantly improves both the diversity and accuracy of the samplings compared with previous state-of-the-art sampling approaches. Code and pre-trained models are available at https://github.com/Droliven/diverse_sampling.