Fast and Efficient Locomotion via Learned Gait Transitions
Not on the map. Why?
Abstract (arXiv)
We focus on the problem of developing energy efficient controllers for quadrupedal robots. Animals can actively switch gaits at different speeds to lower their energy consumption. In this paper, we devise a hierarchical learning framework, in which distinctive locomotion gaits and natural gait transitions emerge automatically with a simple reward of energy minimization. We use evolutionary strategies (ES) to train a high-level gait policy that specifies gait patterns of each foot, while the low-level convex MPC controller optimizes the motor commands so that the robot can walk at a desired velocity using that gait pattern. We test our learning framework on a quadruped robot and demonstrate automatic gait transitions, from walking to trotting and to fly-trotting, as the robot increases its speed. We show that the learned hierarchical controller consumes much less energy across a wide range of locomotion speed than baseline controllers.
Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 27 of its 60 references that are in the map. First posted on arXiv 9 Apr 2021.
Builds on
27 of 60 references.
References · 27
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 555
- 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
- 2019Learning agile and dynamic motor skills for legged robotscited by 402
- 2016The CMA Evolution Strategy: A Tutorialcited by 364
- 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 277
- 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
- 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
- 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
- 2019Data Efficient Reinforcement Learning for Legged Robotscited by 60
- 2018Learning Symmetric and Low-energy Locomotioncited by 53
15 more
- 2019Policies Modulating Trajectory Generatorscited by 47
- 2019Learning Fast Adaptation with Meta Strategy Optimizationcited by 46
- 2016Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?cited by 44
- 2019Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Controlcited by 43
- 2020Rapidly Adaptable Legged Robots via Evolutionary Meta-Learningcited by 42
- 2019Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Realcited by 39
- 2019Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspectivecited by 26
- 2020Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotioncited by 22
- 2020From Pixels to Legs: Hierarchical Learning of Quadruped Locomotioncited by 20
- 2019Feedback MPC for Torque-Controlled Legged Robotscited by 19
- 2020Learning Task Space Actions for Bipedal Locomotioncited by 15
- 2019Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learningcited by 13
- 2017Real-Time Motion Planning of Legged Robots: A Model Predictive Control Approachcited by 9
- 2018Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Bipedcited by 8
- 2018Passive Whole-body Control for Quadruped Robots: Experimental Validation over Challenging Terraincited by 2
Built on by
Citing papers per year: 2021: 6; 2022: 15; 2023: 9; 2024: 9; 2025: 6; 2026: 5.
50 papers.
Sort:
By area:
- 2022DayDreamer: World Models for Physical Robot Learningcited by 372
- 2023Robot Parkour Learningcited by 115
- 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 113
- 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
- 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 71
- 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 68
- 2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robotscited by 67
- 2021Coupling Vision and Proprioception for Navigation of Legged Robotscited by 46
- 2023Prompt a Robot to Walk with Large Language Modelscited by 44
- 2021Learning to Jump from Pixelscited by 39
- 2022Advanced Skills by Learning Locomotion and Local Navigation End-to-Endcited by 39
- 2024Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfercited by 35
38 more
- 2022Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robotcited by 32
- 2024Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robotscited by 32
- 2023Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotioncited by 30
- 2022CPG-RL: Learning Central Pattern Generators for Quadruped Locomotioncited by 24
- 2022Learning Torque Control for Quadrupedal Locomotioncited by 22
- 2024Learning Quadruped Locomotion Using Differentiable Simulationcited by 22
- 2023Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotioncited by 21
- 2022Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomizationcited by 15
- 2023ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robotscited by 15
- 2022Sim-to-Real Transfer for Quadrupedal Locomotion via Terrain Transformercited by 12
- 2024Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushingcited by 9
- 2022Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environmentscited by 7
- 2022Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotioncited by 7
- 2021Learning Quadruped Locomotion Policies using Logical Rulescited by 6
- 2024Gaitor: Learning a Unified Representation Across Gaits for Real-World Quadruped Locomotioncited by 6
- 2022Learning Semantics-Aware Locomotion Skills from Human Demonstrationcited by 5
- 2023Puppeteer and Marionette: Learning Anticipatory Quadrupedal Locomotion Based on Interactions of a Central Pattern Generator and Supraspinal Drivecited by 5
- 2023Continuous Versatile Jumping Using Learned Action Residualscited by 5
- 2025Optimizing Bipedal Locomotion for The 100m Dash With Comparison to Human Runningcited by 3
- 2021Next Steps: Learning a Disentangled Gait Representation for Versatile Quadruped Locomotioncited by 2
- 2022VAE-Loco: Versatile Quadruped Locomotion by Learning a Disentangled Gait Representationcited by 2
- 2023DeepTransition: Viability Leads to the Emergence of Gait Transitions in Learning Anticipatory Quadrupedal Locomotion Skillscited by 2
- 2024An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcementcited by 2
- 2025Real-Time Gait Adaptation for Quadrupeds using Model Predictive Control and Reinforcement Learningcited by 2
- 2026PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamicscited by 2
- 2026Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specificationscited by 2
- 2024Learning Emergent Gaits with Decentralized Phase Oscillators: on the role of Observations, Rewards, and Feedbackcited by 1
- 2025Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learningcited by 1
- 2025QuietPaw: Learning Quadrupedal Locomotion with Versatile Noise Preference Alignmentcited by 1
- 2022Learning Agile Paths from Optimal Controlcited by 0
- 2023Unlock Reliable Skill Inference for Quadruped Adaptive Behavior by Skill Graphcited by 0
- 2024Agile and versatile bipedal robot tracking control through reinforcement learningcited by 0
- 2024Gait Switching and Enhanced Stabilization of Walking Robots with Deep Learning-based Reachability: A Case Study on Two-link Walkercited by 0
- 2025PALo: Learning Posture-Aware Locomotion for Quadruped Robotscited by 0
- 2025Real-DRL: Teach and Learn in Realitycited by 0
- 2026Learning Energy-Efficient Air--Ground Actuation for Hybrid Robots on Stair-Like Terraincited by 0
- 2026LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priorscited by 0
- 2026From LLM-Generated Specifications to Learned Quadruped Locomotioncited by 0
- 2026From LLM-Generated Specifications to Learned Quadruped Locomotion7 Sep 2026
- 2026Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specifications1 Jul 2026
- 2026LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors14 Jun 2026
- 2026PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamics18 May 2026
- 2026Learning Energy-Efficient Air--Ground Actuation for Hybrid Robots on Stair-Like Terrain13 Mar 2026
- 2025Real-DRL: Teach and Learn in Reality30 Oct 2025
- 2025Real-Time Gait Adaptation for Quadrupeds using Model Predictive Control and Reinforcement Learning23 Oct 2025
- 2025Optimizing Bipedal Locomotion for The 100m Dash With Comparison to Human Running5 Aug 2025
- 2025QuietPaw: Learning Quadrupedal Locomotion with Versatile Noise Preference Alignment6 Mar 2025
- 2025PALo: Learning Posture-Aware Locomotion for Quadruped Robots6 Mar 2025
- 2025Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learning6 Feb 2025
- 2024Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing11 Nov 2024
38 more
- 2024Gait Switching and Enhanced Stabilization of Walking Robots with Deep Learning-based Reachability: A Case Study on Two-link Walker10 Sep 2024
- 2024Gaitor: Learning a Unified Representation Across Gaits for Real-World Quadruped Locomotion29 May 2024
- 2024Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots3 May 2024
- 2024Agile and versatile bipedal robot tracking control through reinforcement learning12 Apr 2024
- 2024Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer8 Apr 2024
- 2024Learning Quadruped Locomotion Using Differentiable Simulation21 Mar 2024
- 2024An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement4 Mar 2024
- 2024Learning Emergent Gaits with Decentralized Phase Oscillators: on the role of Observations, Rewards, and Feedback13 Feb 2024
- 2023Unlock Reliable Skill Inference for Quadruped Adaptive Behavior by Skill Graph10 Nov 2023
- 2023Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion26 Oct 2023
- 2023ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robots16 Oct 2023
- 2023Prompt a Robot to Walk with Large Language Models18 Sep 2023
- 2023Robot Parkour Learning11 Sep 2023
- 2023DeepTransition: Viability Leads to the Emergence of Gait Transitions in Learning Anticipatory Quadrupedal Locomotion Skills12 Jun 2023
- 2023Continuous Versatile Jumping Using Learned Action Residuals17 Apr 2023
- 2023Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion20 Mar 2023
- 2023Puppeteer and Marionette: Learning Anticipatory Quadrupedal Locomotion Based on Interactions of a Central Pattern Generator and Supraspinal Drive26 Feb 2023
- 2022Sim-to-Real Transfer for Quadrupedal Locomotion via Terrain Transformer15 Dec 2022
- 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior6 Dec 2022
- 2022Learning Agile Paths from Optimal Control30 Nov 2022
- 2022CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion1 Nov 2022
- 2022Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion29 Sep 2022
- 2022Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomization26 Sep 2022
- 2022Advanced Skills by Learning Locomotion and Local Navigation End-to-End26 Sep 2022
- 2022Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environments19 Sep 2022
- 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning16 Aug 2022
- 2022Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robot1 Aug 2022
- 2022DayDreamer: World Models for Physical Robot Learning28 Jun 2022
- 2022Learning Semantics-Aware Locomotion Skills from Human Demonstration27 Jun 2022
- 2022VAE-Loco: Versatile Quadruped Locomotion by Learning a Disentangled Gait Representation2 May 2022
- 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions28 Mar 2022
- 2022Learning Torque Control for Quadrupedal Locomotion10 Mar 2022
- 2021Next Steps: Learning a Disentangled Gait Representation for Versatile Quadruped Locomotion9 Dec 2021
- 2021Coupling Vision and Proprioception for Navigation of Legged Robots3 Dec 2021
- 2021Learning to Jump from Pixels28 Oct 2021
- 2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots25 Oct 2021
- 2021Learning Quadruped Locomotion Policies using Logical Rules23 Jul 2021
- 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers8 Jul 2021
- 2022DayDreamer: World Models for Physical Robot Learningcited by 372
- 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learningcited by 76
- 2024An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcementcited by 2
- 2026PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamicscited by 2
- 2026PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamics18 May 2026
- 2024An Efficient Model-Based Approach on Learning Agile Motor Skills without Reinforcement4 Mar 2024
- 2022A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning16 Aug 2022
- 2022DayDreamer: World Models for Physical Robot Learning28 Jun 2022
- 2023Robot Parkour Learningcited by 115
- 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 113
- 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 71
- 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 68
- 2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robotscited by 67
- 2021Coupling Vision and Proprioception for Navigation of Legged Robotscited by 46
- 2023Prompt a Robot to Walk with Large Language Modelscited by 44
- 2021Learning to Jump from Pixelscited by 39
- 2022Advanced Skills by Learning Locomotion and Local Navigation End-to-Endcited by 39
- 2024Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfercited by 35
- 2022Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robotcited by 32
- 2024Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robotscited by 32
12 most cited of 46 All 46 (JSON)
- 2026From LLM-Generated Specifications to Learned Quadruped Locomotion7 Sep 2026
- 2026Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specifications1 Jul 2026
- 2026LoComposition: Terrain-Adaptive Energy-Efficient Quadruped Locomotion without Gait Priors14 Jun 2026
- 2026Learning Energy-Efficient Air--Ground Actuation for Hybrid Robots on Stair-Like Terrain13 Mar 2026
- 2025Real-DRL: Teach and Learn in Reality30 Oct 2025
- 2025Real-Time Gait Adaptation for Quadrupeds using Model Predictive Control and Reinforcement Learning23 Oct 2025
- 2025Optimizing Bipedal Locomotion for The 100m Dash With Comparison to Human Running5 Aug 2025
- 2025QuietPaw: Learning Quadrupedal Locomotion with Versatile Noise Preference Alignment6 Mar 2025
- 2025PALo: Learning Posture-Aware Locomotion for Quadruped Robots6 Mar 2025
- 2025Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learning6 Feb 2025
- 2024Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing11 Nov 2024
- 2024Gait Switching and Enhanced Stabilization of Walking Robots with Deep Learning-based Reachability: A Case Study on Two-link Walker10 Sep 2024
12 newest of 46 All 46 (JSON)