mlsanityData to 7 Oct 2026

Custom Sine Waves Are Enough for Imitation Learning of Bipedal Gaits with Different Styles

arXiv 2204.04157 ↗First posted 8 Apr 2022cs.ROcited by 0

On the map: Offline RL and imitation learning from data → Real-world robot offline RL

Abstract (arXiv)

Not until recently, robust bipedal locomotion has been achieved through reinforcement learning. However, existing implementations rely heavily on insights and efforts from human experts, which is costly for the iterative design of robot systems. Also, styles of the learned motion are strictly limited to that of the reference. In this paper, we propose a new way to learn bipedal locomotion from a simple sine wave as the reference for foot heights. With the naive human insight that the two feet should be lifted up alternatively and periodically, we experimentally demonstrate on the Cassie robot that, a simple reward function is able to make the robot learn to walk end-to-end and efficiently without any explicit knowledge of the model. With custom sine waves, the learned gait pattern can also have customized styles. Codes are released at github.com/WooQi57/sin-cassie-rl.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 30 references that are in the map. First posted on arXiv 8 Apr 2022.

Builds on

17 of 30 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  4. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  5. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  6. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
  7. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  8. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 95
  9. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 73
  10. 2020Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Compositioncited by 71
  11. 2016Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?cited by 44
  12. 2018Feedback Control For Cassie With Deep Reinforcement Learningcited by 36
5 more
  1. 2020Learning Task Space Actions for Bipedal Locomotioncited by 15
  2. 2021DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learningcited by 15
  3. 2020Learning Spring Mass Locomotion: Guiding Policies with a Reduced-Order Modelcited by 8
  4. 2021Accessibility-Based Clustering for Efficient Learning of Locomotion Skillscited by 5
  5. 2022Bayesian Optimization Meets Hybrid Zero Dynamics: Safe Parameter Learning for Bipedal Locomotion Controlcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Adaptive Mimic: Deep Reinforcement Learning of Parameterized Bipedal Walking from Infeasible References ↗
  2. 2020Learning natural locomotion behaviors for humanoid robots using human knowledge
  3. 2023Infer and Adapt: Bipedal Locomotion Reward Learning from Demonstrations via Inverse Reinforcement Learning
  4. 2025Think on your feet: Seamless Transition between Human-like Locomotion in Response to Changing Commands
  5. 2019Self-Imitation Learning of Locomotion Movements through Termination Curriculum
  6. 2026Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking
  7. 2023LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotion
  8. 2024Exciting Action: Investigating Efficient Exploration for Learning Musculoskeletal Humanoid Locomotion