mlsanityData to 7 Oct 2026

Emergence of Locomotion Behaviours in Rich Environments

arXiv 1707.02286 ↗First posted 7 Jul 2017cs.AIcited by 159

Not on the map. Why?

Abstract (arXiv)

The reinforcement learning paradigm allows, in principle, for complex behaviours to be learned directly from simple reward signals. In practice, however, it is common to carefully hand-design the reward function to encourage a particular solution, or to derive it from demonstration data. In this paper explore how a rich environment can help to promote the learning of complex behavior. Specifically, we train agents in diverse environmental contexts, and find that this encourages the emergence of robust behaviours that perform well across a suite of tasks. We demonstrate this principle for locomotion -- behaviours that are known for their sensitivity to the choice of reward. We train several simulated bodies on a diverse set of challenging terrains and obstacles, using a simple reward function based on forward progress. Using a novel scalable variant of policy gradient reinforcement learning, our agents learn to run, jump, crouch and turn as required by the environment without explicit reward-based guidance. A visual depiction of highlights of the learned behavior can be viewed following https://youtu.be/hx_bgoTF7bs .

Key facts. Cited by 159 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 2; 2021: 52; 2022: 31; 2023: 26; 2024: 15; 2025: 16; 2026: 15.

159 papers.

Sort:

By area:

  1. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  2. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  3. 2021Character Controllers Using Motion VAEscited by 182
  4. 2019Goal-conditioned Imitation Learningcited by 159
  5. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  6. 2021Embodied Intelligence via Learning and Evolutioncited by 154
  7. 2021The Benchmark Lotterycited by 133
  8. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  9. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
  10. 2023Synthesizing Physical Character-Scene Interactionscited by 102
  11. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  12. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
38 more
  1. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 71
  2. 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 68
  3. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
  4. 2021Offline Meta-Reinforcement Learning with Online Self-Supervisioncited by 52
  5. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  6. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 50
  7. 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
  8. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 49
  9. 2021Hierarchical Skills for Efficient Explorationcited by 37
  10. 2022Trajectory Optimization for Physics-Based Reconstruction of 3d Human Pose from Monocular Videocited by 37
  11. 2023Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotioncited by 33
  12. 2025BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholdscited by 32
  13. 2025FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Controlcited by 32
  14. 2021Distributed Heuristic Multi-Agent Path Finding with Communicationcited by 31
  15. 2023RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learningcited by 29
  16. 2022Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviorscited by 27
  17. 2022Towards Versatile Embodied Navigationcited by 26
  18. 2022MoCapAct: A Multi-Task Dataset for Simulated Humanoid Controlcited by 24
  19. 2024Hierarchical World Models as Visual Whole-Body Humanoid Controllerscited by 24
  20. 2021Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Surveycited by 23
  21. 2022GANimator: Neural Motion Synthesis from a Single Sequencecited by 23
  22. 2022Learning Options via Compressioncited by 22
  23. 2023Latent Exploration for Reinforcement Learningcited by 21
  24. 2021Combining Physics and Deep Learning to learn Continuous-Time Dynamics Modelscited by 20
  25. 2022You May Not Need Ratio Clipping in PPOcited by 19
  26. 2021Learning Modular Robot Control Policiescited by 18
  27. 2021Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomizationcited by 18
  28. 2022Low-Rank Modular Reinforcement Learning via Muscle Synergycited by 16
  29. 2022Dungeons and Data: A Large-Scale NetHack Datasetcited by 16
  30. 2021Discovering Diverse Athletic Jumping Strategiescited by 15
  31. 2021Evaluating model-based planning and planner amortization for continuous controlcited by 15
  32. 2023AdaptNet: Policy Adaptation for Physics-Based Character Controlcited by 15
  33. 2024Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Criticscited by 13
  34. 2021A GAN-Like Approach for Physics-Based Imitation Learning and Interactive Character Controlcited by 12
  35. 2022Generative GaitNetcited by 11
  36. 2022Learning to Get Upcited by 11
  37. 2023H-GAP: Humanoid Control with a Generalist Plannercited by 11
  38. 2025Learning Sim-to-Real Humanoid Locomotion in 15 Minutescited by 11

50 most cited of 159 All 159 (JSON)