# Motron: Multimodal Probabilistic Human Motion Forecasting

arXiv: https://arxiv.org/abs/2203.04132 · HTML: https://mlsanity.com/paper/2203.04132/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 56 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks](https://mlsanity.com/paper/1907.03395/) (2019)
- [Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1611.02648/) (2016)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)
- [Multiple Futures Prediction](https://mlsanity.com/paper/1911.00997/) (2019)
- [HP-GAN: Probabilistic 3D human motion prediction via GAN](https://mlsanity.com/paper/1711.09561/) (2017)
- [History Repeats Itself: Human Motion Prediction via Motion Attention](https://mlsanity.com/paper/2007.11755/) (2020)
- [A Spatio-temporal Transformer for 3D Human Motion Prediction](https://mlsanity.com/paper/2004.08692/) (2020)
- [Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction](https://mlsanity.com/paper/2003.08802/) (2020)
- [We are More than Our Joints: Predicting how 3D Bodies Move](https://mlsanity.com/paper/2012.00619/) (2020)
- [The Pose Knows: Video Forecasting by Generating Pose Futures](https://mlsanity.com/paper/1705.00053/) (2017)
- [MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics](https://mlsanity.com/paper/1808.04545/) (2018)
- [Diverse Trajectory Forecasting with Determinantal Point Processes](https://mlsanity.com/paper/1907.04967/) (2019)
- [DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data](https://mlsanity.com/paper/1706.02071/) (2017)
- [Learning Human Motion Models for Long-term Predictions](https://mlsanity.com/paper/1704.02827/) (2017)
- [Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective](https://mlsanity.com/paper/1806.07772/) (2018)
- [BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN](https://mlsanity.com/paper/1812.02591/) (2018)
- [Modeling Human Motion with Quaternion-based Neural Networks](https://mlsanity.com/paper/1901.07677/) (2019)
- [Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic](https://mlsanity.com/paper/1805.02513/) (2018)
- [A Smooth Representation of Belief over SO(3) for Deep Rotation Learning with Uncertainty](https://mlsanity.com/paper/2006.01031/) (2020)
- [Generative Modeling of Multimodal Multi-Human Behavior](https://mlsanity.com/paper/1803.02015/) (2018)

## Built on by (most cited first, 30 of 30)

- [BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction](https://mlsanity.com/paper/2211.14304/) (2022)
- [HumanMAC: Masked Motion Completion for Human Motion Prediction](https://mlsanity.com/paper/2302.03665/) (2023)
- [Seamless Human Motion Composition with Blended Positional Encodings](https://mlsanity.com/paper/2402.15509/) (2024)
- [SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation](https://mlsanity.com/paper/2304.10417/) (2023)
- [Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion](https://mlsanity.com/paper/2312.04466/) (2023)
- [Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchors](https://mlsanity.com/paper/2302.04860/) (2023)
- [A generic diffusion-based approach for 3D human pose prediction in the wild](https://mlsanity.com/paper/2210.05669/) (2022)
- [AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agents](https://mlsanity.com/paper/2403.12835/) (2024)
- [Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioning](https://mlsanity.com/paper/2404.05218/) (2024)
- [Interactive Humanoid: Online Full-Body Motion Reaction Synthesis with Social Affordance Canonicalization and Forecasting](https://mlsanity.com/paper/2312.08983/) (2023)
- [Robots That Can See: Leveraging Human Pose for Trajectory Prediction](https://mlsanity.com/paper/2309.17209/) (2023)
- [CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion](https://mlsanity.com/paper/2305.12554/) (2023)
- [Auxiliary Tasks Benefit 3D Skeleton-based Human Motion Prediction](https://mlsanity.com/paper/2308.08942/) (2023)
- [Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Prediction](https://mlsanity.com/paper/2501.06035/) (2025)
- [Toward Reliable Human Pose Forecasting with Uncertainty](https://mlsanity.com/paper/2304.06707/) (2023)
- [MotionWavelet: Human Motion Prediction via Wavelet Manifold Learning](https://mlsanity.com/paper/2411.16964/) (2024)
- [Normalizing Flows on the Product Space of SO(3) Manifolds for Probabilistic Human Pose Modeling](https://mlsanity.com/paper/2404.05675/) (2024)
- [MotionMap: Representing Multimodality in Human Pose Forecasting](https://mlsanity.com/paper/2412.18883/) (2024)
- [FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations](https://mlsanity.com/paper/2211.14309/) (2022)
- [DeFeeNet: Consecutive 3D Human Motion Prediction with Deviation Feedback](https://mlsanity.com/paper/2304.04496/) (2023)
- [CacheFlow: Fast Human Motion Prediction by Cached Normalizing Flow](https://mlsanity.com/paper/2505.13140/) (2025)
- [Uncertainty-aware Probabilistic 3D Human Motion Forecasting via Invertible Networks](https://mlsanity.com/paper/2507.14694/) (2025)
- [MMCM: Multimodality-aware Metric using Clustering-based Modes for Probabilistic Human Motion Prediction](https://mlsanity.com/paper/2511.15179/) (2025)
- [3D Pose Nowcasting: Forecast the Future to Improve the Present](https://mlsanity.com/paper/2308.12914/) (2023)
- [Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic](https://mlsanity.com/paper/2507.04062/) (2025)
- [Cross-Modal Alignment via Variational Copula Modelling](https://mlsanity.com/paper/2511.03196/) (2025)
- [SMamDiff: Spatial Mamba for Stochastic Human Motion Prediction](https://mlsanity.com/paper/2512.00355/) (2025)
- [Capturing Uncertainty in Human Motion for Representation Learning in Soccer](https://mlsanity.com/paper/2608.11203/) (2026)
- [Gaussian-Mixture Latent Flow for Stochastic 3D Human Motion Prediction](https://mlsanity.com/paper/2608.21093/) (2026)
- [Probabilistic electrical power demand forecasting with uncertainty quantification](https://mlsanity.com/paper/2609.34120/) (2026)

## Abstract (arXiv)

Autonomous systems and humans are increasingly sharing the same space. Robots work side by side or even hand in hand with humans to balance each other's limitations. Such cooperative interactions are ever more sophisticated. Thus, the ability to reason not just about a human's center of gravity position, but also its granular motion is an important prerequisite for human-robot interaction. Though, many algorithms ignore the multimodal nature of humans or neglect uncertainty in their motion forecasts. We present Motron, a multimodal, probabilistic, graph-structured model, that captures human's multimodality using probabilistic methods while being able to output deterministic maximum-likelihood motions and corresponding confidence values for each mode. Our model aims to be tightly integrated with the robotic planning-control-interaction loop; outputting physically feasible human motions and being computationally efficient. We demonstrate the performance of our model on several challenging real-world motion forecasting datasets, outperforming a wide array of generative/variational methods while providing state-of-the-art single-output motions if required. Both using significantly less computational power than state-of-the art algorithms.