mlsanityData to 7 Oct 2026

Character Controllers Using Motion VAEs

arXiv 2103.14274 ↗First posted 26 Mar 2021cs.LG cs.GRcited by 182

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

A fundamental problem in computer animation is that of realizing purposeful and realistic human movement given a sufficiently-rich set of motion capture clips. We learn data-driven generative models of human movement using autoregressive conditional variational autoencoders, or Motion VAEs. The latent variables of the learned autoencoder define the action space for the movement and thereby govern its evolution over time. Planning or control algorithms can then use this action space to generate desired motions. In particular, we use deep reinforcement learning to learn controllers that achieve goal-directed movements. We demonstrate the effectiveness of the approach on multiple tasks. We further evaluate system-design choices and describe the current limitations of Motion VAEs.

Key facts. Cited by 182 papers in the map (data through 7 Oct 2026). Builds on 16 of its 44 references that are in the map. First posted on arXiv 26 Mar 2021.

Builds on

16 of 44 references.

References · 16

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2016OpenAI Gymcited by 1,684
  5. 2018World Modelscited by 1,444
  6. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  7. 2013Generating Sequences With Recurrent Neural Networkscited by 549
  8. 2017On human motion prediction using recurrent neural networkscited by 336
  9. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  10. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
  11. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  12. 2017Meta Learning Shared Hierarchiescited by 93
4 more
  1. 2018MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamicscited by 86
  2. 2018Learning Symmetric and Low-energy Locomotioncited by 53
  3. 2018Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Controlcited by 39
  4. 2019Synthesis of Biologically Realistic Human Motion Using Joint Torque Actuationcited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 34; 2023: 47; 2024: 42; 2025: 27; 2026: 13.

182 papers.

Sort:

By area:

  1. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  2. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  3. 2022EDGE: Editable Dance Generation From Musiccited by 284
  4. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  5. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
  6. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  7. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  8. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 188
  9. 2022TEACH: Temporal Action Composition for 3D Humanscited by 173
  10. 2023Decoupling Human and Camera Motion from Videos in the Wildcited by 140
  11. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 137
  12. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 134
38 more
  1. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  2. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 121
  3. 2023AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanismcited by 116
  4. 2021Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoderscited by 109
  5. 2023Synthesizing Diverse Human Motions in 3D Indoor Scenescited by 109
  6. 2023Universal Humanoid Motion Representations for Physics-Based Controlcited by 109
  7. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 109
  8. 2021Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimationcited by 101
  9. 2023CALM: Conditional Adversarial Latent Models for Directable Virtual Characterscited by 91
  10. 2021SAGA: Stochastic Whole-Body Grasping with Contactcited by 89
  11. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 80
  12. 2023CIRCLE: Capture In Rich Contextual Environmentscited by 79
  13. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 79
  14. 2022ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generationcited by 77
  15. 2024ExBody2: Advanced Expressive Humanoid Whole-Body Controlcited by 75
  16. 2022QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatarscited by 74
  17. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 72
  18. 2022Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phasescited by 71
  19. 2024Flexible Motion In-betweening with Diffusion Modelscited by 69
  20. 2021Generating Smooth Pose Sequences for Diverse Human Motion Predictioncited by 66
  21. 2022NeMF: Neural Motion Fields for Kinematic Animationcited by 66
  22. 2023A Comprehensive Review of Data-Driven Co-Speech Gesture Generationcited by 66
  23. 2021Physics-based Human Motion Estimation and Synthesis from Videoscited by 65
  24. 2021The Wanderings of Odysseus in 3D Scenescited by 59
  25. 2023MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representationscited by 58
  26. 2022GFPose: Learning 3D Human Pose Prior with Gradient Fieldscited by 57
  27. 2024Taming Diffusion Probabilistic Models for Character Controlcited by 56
  28. 2024Estimating Body and Hand Motion in an Ego-sensed Worldcited by 56
  29. 2022AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatarscited by 55
  30. 2022FLAG: Flow-based 3D Avatar Generation from Sparse Observationscited by 54
  31. 2021Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structurecited by 53
  32. 2023Interactive Character Control with Auto-Regressive Motion Diffusion Modelscited by 52
  33. 2023Locomotion-Action-Manipulation: Synthesizing Human-Scene Interactions in Complex 3D Environmentscited by 51
  34. 2024Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Cameracited by 50
  35. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  36. 2022Spatio-Temporal Gating-Adjacency GCN for Human Motion Predictioncited by 48
  37. 2023TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesiscited by 48
  38. 2023PADL: Language-Directed Physics-Based Character Controlcited by 47

50 most cited of 182 All 182 (JSON)