mlsanityData to 7 Oct 2026

MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamics

arXiv 1808.04545 ↗First posted 14 Aug 2018cs.LG cs.AI cs.CVcited by 86

Not on the map. Why?

Abstract (arXiv)

Long-term human motion can be represented as a series of motion modes---motion sequences that capture short-term temporal dynamics---with transitions between them. We leverage this structure and present a novel Motion Transformation Variational Auto-Encoders (MT-VAE) for learning motion sequence generation. Our model jointly learns a feature embedding for motion modes (that the motion sequence can be reconstructed from) and a feature transformation that represents the transition of one motion mode to the next motion mode. Our model is able to generate multiple diverse and plausible motion sequences in the future from the same input. We apply our approach to both facial and full body motion, and demonstrate applications like analogy-based motion transfer and video synthesis.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 19; 2022: 23; 2023: 16; 2024: 12; 2025: 10; 2026: 5.

86 papers.

Sort:

By area:

  1. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  2. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  3. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
  4. 2021Character Controllers Using Motion VAEscited by 182
  5. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 151
  6. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
  7. 2021Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoderscited by 109
  8. 2021SAGA: Stochastic Whole-Body Grasping with Contactcited by 89
  9. 2022BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Predictioncited by 82
  10. 2023HumanMAC: Masked Motion Completion for Human Motion Predictioncited by 81
  11. 2022ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generationcited by 77
  12. 2021Multi-Person Extreme Motion Predictioncited by 74
38 more
  1. 2021Generating Smooth Pose Sequences for Diverse Human Motion Predictioncited by 66
  2. 2021Physics-based Human Motion Estimation and Synthesis from Videoscited by 65
  3. 2021The Wanderings of Odysseus in 3D Scenescited by 59
  4. 2022Uncertainty Inspired Underwater Image Enhancementcited by 53
  5. 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
  6. 2019Imitation Learning for Human Pose Predictioncited by 44
  7. 2023Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Modelscited by 43
  8. 2023Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchorscited by 34
  9. 2021Task-Generic Hierarchical Human Motion Prior using VAEscited by 33
  10. 2022A generic diffusion-based approach for 3D human pose prediction in the wildcited by 33
  11. 2023PACE: Human and Camera Motion Estimation from in-the-wild Videoscited by 33
  12. 2022Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an Auxiliary Spacecited by 32
  13. 2022Motron: Multimodal Probabilistic Human Motion Forecastingcited by 30
  14. 2022Human Joint Kinematics Diffusion-Refinement for Stochastic Motion Predictioncited by 28
  15. 2023Stochastic Multi-Person 3D Motion Forecastingcited by 28
  16. 2022Weakly-supervised Action Transition Learning for Stochastic Human Motion Predictioncited by 26
  17. 2022GLENet: Boosting 3D Object Detectors with Generative Label Uncertainty Estimationcited by 24
  18. 2021Aggregated Multi-GANs for Controlled 3D Human Motion Predictioncited by 23
  19. 2021Revisiting Hierarchical Approach for Persistent Long-Term Video Predictioncited by 23
  20. 2023HMP: Hand Motion Priors for Pose and Shape Estimation from Videocited by 23
  21. 2023InsActor: Instruction-driven Physics-based Characterscited by 23
  22. 2024M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Modelscited by 23
  23. 2024Harmon: Whole-Body Motion Generation of Humanoid Robots from Language Descriptionscited by 22
  24. 2023CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusioncited by 21
  25. 2022Action-conditioned On-demand Motion Generationcited by 18
  26. 2022MIMO Is All You Need : A Strong Multi-In-Multi-Out Baseline for Video Predictioncited by 17
  27. 2022HiT-DVAE: Human Motion Generation via Hierarchical Transformer Dynamical VAEcited by 16
  28. 2023TransFusion: A Practical and Effective Transformer-based Diffusion Model for 3D Human Motion Predictioncited by 16
  29. 2021Action2video: Generating Videos of Human 3D Actionscited by 15
  30. 2024COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimationcited by 15
  31. 2021Behavior-Driven Synthesis of Human Dynamicscited by 14
  32. 2021TrajeVAE: Controllable Human Motion Generation from Trajectoriescited by 14
  33. 2023Can We Use Diffusion Probabilistic Models for 3D Motion Prediction?cited by 14
  34. 2024Learning Semantic Latent Directions for Accurate and Controllable Human Motion Predictioncited by 14
  35. 2024Sitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenescited by 14
  36. 2025Simulating the Real World: A Unified Survey of Multimodal Generative Modelscited by 14
  37. 2022MotionAug: Augmentation with Physical Correction for Human Motion Predictioncited by 13
  38. 2025Dynamic Motion Blending for Versatile Motion Editingcited by 13

50 most cited of 86 All 86 (JSON)