# DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills

arXiv: https://arxiv.org/abs/1804.02717 · HTML: https://mlsanity.com/paper/1804.02717/

## Key facts

- Cited by 163 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2018.

## Built on by (most cited first, 30 of 163)

- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [Perpetual Humanoid Control for Real-time Simulated Avatars](https://mlsanity.com/paper/2305.06456/) (2023)
- [Object Motion Guided Human Motion Synthesis](https://mlsanity.com/paper/2309.16237/) (2023)
- [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021)
- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion](https://mlsanity.com/paper/2508.08241/) (2025)
- [Goal-conditioned Imitation Learning](https://mlsanity.com/paper/1906.05838/) (2019)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation](https://mlsanity.com/paper/2403.04436/) (2024)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [Universal Humanoid Motion Representations for Physics-Based Control](https://mlsanity.com/paper/2310.04582/) (2023)
- [MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting](https://mlsanity.com/paper/2409.14393/) (2024)
- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [Synthesizing Physical Character-Scene Interactions](https://mlsanity.com/paper/2302.00883/) (2023)
- [3D Human Pose Estimation via Intuitive Physics](https://mlsanity.com/paper/2303.18246/) (2023)
- [Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation](https://mlsanity.com/paper/2106.05969/) (2021)
- [CALM: Conditional Adversarial Latent Models for Directable Virtual Characters](https://mlsanity.com/paper/2305.02195/) (2023)
- [State-Only Imitation Learning for Dexterous Manipulation](https://mlsanity.com/paper/2004.04650/) (2020)
- [From Motor Control to Team Play in Simulated Humanoid Football](https://mlsanity.com/paper/2105.12196/) (2021)
- [QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars](https://mlsanity.com/paper/2209.09391/) (2022)
- [PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction](https://mlsanity.com/paper/2312.04393/) (2023)
- [Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phases](https://mlsanity.com/paper/2201.04439/) (2022)
- [Neural Monocular 3D Human Motion Capture with Physical Awareness](https://mlsanity.com/paper/2105.01057/) (2021)
- [Visual Imitation Enables Contextual Humanoid Control](https://mlsanity.com/paper/2505.03729/) (2025)
- [DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment](https://mlsanity.com/paper/2307.00329/) (2023)
- [MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations](https://mlsanity.com/paper/2310.10198/) (2023)
- [Taming Diffusion Probabilistic Models for Character Control](https://mlsanity.com/paper/2404.15121/) (2024)
- [DTC: Deep Tracking Control](https://mlsanity.com/paper/2309.15462/) (2023)
- [Interactive Character Control with Auto-Regressive Motion Diffusion Models](https://mlsanity.com/paper/2306.00416/) (2023)

## Abstract (arXiv)

A longstanding goal in character animation is to combine data-driven specification of behavior with a system that can execute a similar behavior in a physical simulation, thus enabling realistic responses to perturbations and environmental variation. We show that well-known reinforcement learning (RL) methods can be adapted to learn robust control policies capable of imitating a broad range of example motion clips, while also learning complex recoveries, adapting to changes in morphology, and accomplishing user-specified goals. Our method handles keyframed motions, highly-dynamic actions such as motion-captured flips and spins, and retargeted motions. By combining a motion-imitation objective with a task objective, we can train characters that react intelligently in interactive settings, e.g., by walking in a desired direction or throwing a ball at a user-specified target. This approach thus combines the convenience and motion quality of using motion clips to define the desired style and appearance, with the flexibility and generality afforded by RL methods and physics-based animation. We further explore a number of methods for integrating multiple clips into the learning process to develop multi-skilled agents capable of performing a rich repertoire of diverse skills. We demonstrate results using multiple characters (human, Atlas robot, bipedal dinosaur, dragon) and a large variety of skills, including locomotion, acrobatics, and martial arts.