mlsanityData to 7 Oct 2026

Self-Imitation Learning of Locomotion Movements through Termination Curriculum

arXiv 1907.11842 ↗First posted 27 Jul 2019cs.GR cs.LG cs.ROcited by 0

On the map: Offline RL and imitation learning from data → Real-world robot offline RL

Abstract (arXiv)

Animation and machine learning research have shown great advancements in the past decade, leading to robust and powerful methods for learning complex physically-based animations. However, learning can take hours or days, especially if no reference movement data is available. In this paper, we propose and evaluate a novel combination of techniques for accelerating the learning of stable locomotion movements through self-imitation learning of synthetic animations. First, we produce synthetic and cyclic reference movement using a recent online tree search approach that can discover stable walking gaits in a few minutes. This allows us to use reinforcement learning with Reference State Initialization (RSI) to find a neural network controller for imitating the synthesized reference motion. We further accelerate the learning using a novel curriculum learning approach called Termination Curriculum (TC), that adapts the episode termination threshold over time. The combination of the RSI and TC ensures that simulation budget is not wasted in regions of the state space not visited by the final policy. As a result, our agents can learn locomotion skills in just a few hours on a modest 4-core computer. We demonstrate this by producing locomotion movements for a variety of characters.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 13 of its 47 references that are in the map. First posted on arXiv 27 Jul 2019.

Builds on

13 of 47 references.

References · 13

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  6. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
  7. 2018Maximum a Posteriori Policy Optimisationcited by 318
  8. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  9. 2018Learning Symmetric and Low-energy Locomotioncited by 53
  10. 2018PPO-CMA: Proximal Policy Optimization with Covariance Matrix Adaptationcited by 14
  11. 2018Progressive Reinforcement Learning with Distillation for Multi-Skilled Motion Controlcited by 12
  12. 2016Rolling Horizon Coevolutionary Planning for Two-Player Video Gamescited by 3
1 more
  1. 2018Intelligent Middle-Level Game Controlcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotion
  2. 2026FetchMan: Learning Visual Humanoid Loco-Manipulation Policies from Simulated Experiences ↗
  3. 2023DecAP: Decaying Action Priors for Accelerated Imitation Learning of Torque-Based Legged Locomotion Policies
  4. 2026Learning Quadruped Walking from Seconds of Demonstration
  5. 2024Learning Speed-Adaptive Walking Agent Using Imitation Learning with Physics-Informed Simulation
  6. 2024Exciting Action: Investigating Efficient Exploration for Learning Musculoskeletal Humanoid Locomotion
  7. 2025Walk Like Dogs: Learning Steerable Imitation Controllers for Legged Robots from Unlabeled Motion Data
  8. 2024PDP: Physics-Based Character Animation via Diffusion Policy