mlsanityData to 14 Oct 2026

ZSL-RPPO: Zero-Shot Learning for Quadrupedal Locomotion in Challenging Terrains using Recurrent Proximal Policy Optimization

arXiv 2403.01928 ↗First posted 4 Mar 2024cs.ROcited by 4

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

We present ZSL-RPPO, an improved zero-shot learning architecture that overcomes the limitations of teacher-student neural networks and enables generating robust, reliable, and versatile locomotion for quadrupedal robots in challenging terrains. We propose a new algorithm RPPO (Recurrent Proximal Policy Optimization) that directly trains recurrent neural network in partially observable environments and results in more robust training using domain randomization. Our locomotion controller supports extensive perturbation across simulation-to-reality transfer for both intrinsic and extrinsic physical parameters without further fine-tuning. This can avoid the significant decline of student's performance during simulation-to-reality transfer and therefore enhance the robustness and generalization of the locomotion controller. We deployed our controller on the Unitree A1 and Aliengo robots in real environment and exteroceptive perception is provided by either a solid-state Lidar or a depth camera. Our locomotion controller was tested in various challenging terrains like slippery surfaces, Grassy Terrain, and stairs. Our experiment results and comparison show that our approach significantly outperforms the state-of-the-art.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 39 references that are in the map. First posted on arXiv 4 Mar 2024.

Builds on

23 of 39 references.

References · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  4. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  5. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  6. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  7. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  8. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  9. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  10. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  11. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  12. 2019Active Domain Randomizationcited by 170
11 more
  1. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 151
  2. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
  3. 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 83
  4. 2019Wasserstein Robust Reinforcement Learningcited by 54
  5. 2019Policies Modulating Trajectory Generatorscited by 49
  6. 2023Fast Traversability Estimation for Wild Visual Navigationcited by 44
  7. 2019Distributionally Robust Reinforcement Learningcited by 42
  8. 2020SAMBA: Safe Model-Based & Active Reinforcement Learningcited by 30
  9. 2020Perspectives on Sim2Real Transfer for Robotics: A Summary of the R:SS 2020 Workshopcited by 25
  10. 2020Robust Reinforcement Learning with Wasserstein Constraintcited by 21
  11. 2023Learning-based Design and Control for Quadrupedal Robots with Parallel-Elastic Actuatorscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025RLCAD: Reinforcement Learning Training Gym for Revolution Involved CAD Command Sequence Generationcited by 7
  2. 2024Reinforcement Learning For Quadrupedal Locomotion: Current Advancements And Future Perspectivescited by 2
  3. 2025McARL:Morphology-Control-Aware Reinforcement Learning for Generalizable Quadrupedal Locomotioncited by 1
  4. 2026SRL: Combining SLIP Model and Reinforcement Learning for Agile Robotic Jumpingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Agile perceptive multi-skill locomotion for quadrupedal robots in the wild
  2. 2026Neuromorphic Reinforcement Learning for Quadruped Locomotion Control on Uneven Terrain
  3. 2025Robust Reinforcement Learning-Based Locomotion for Resource-Constrained Quadrupeds with Exteroceptive Sensing ↗
  4. 2025GRoQ-LoCO: Generalist and Robot-agnostic Quadruped Locomotion Control using Offline Datasets
  5. 2021Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner
  6. 2025QuadKAN: KAN-Enhanced Quadruped Motion Control via End-to-End Reinforcement Learning
  7. 2026Learning Locomotion on Complex Terrain for Quadrupedal Robots with Foot Position Maps and Stability Rewards
  8. 2026Towards Torque-Driven Reinforcement Learning for Quadruped Locomotion