# Fast and Efficient Locomotion via Learned Gait Transitions

arXiv: https://arxiv.org/abs/2104.04644 · HTML: https://mlsanity.com/paper/2104.04644/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 60 references that are in the map.
- First posted on arXiv 9 Apr 2021.

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [The CMA Evolution Strategy: A Tutorial](https://mlsanity.com/paper/1604.00772/) (2016)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [Emergence of Locomotion Behaviours in Rich Environments](https://mlsanity.com/paper/1707.02286/) (2017)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Data Efficient Reinforcement Learning for Legged Robots](https://mlsanity.com/paper/1907.03613/) (2019)
- [Learning Symmetric and Low-energy Locomotion](https://mlsanity.com/paper/1801.08093/) (2018)
- [Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control](https://mlsanity.com/paper/1909.06586/) (2019)
- [Policies Modulating Trajectory Generators](https://mlsanity.com/paper/1910.02812/) (2019)
- [Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?](https://mlsanity.com/paper/1611.01055/) (2016)
- [Learning Fast Adaptation with Meta Strategy Optimization](https://mlsanity.com/paper/1909.12995/) (2019)
- [Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning](https://mlsanity.com/paper/2003.01239/) (2020)
- [Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real](https://mlsanity.com/paper/1908.05224/) (2019)
- [Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspective](https://mlsanity.com/paper/1901.11503/) (2019)
- [Learning a Contact-Adaptive Controller for Robust, Efficient Legged Locomotion](https://mlsanity.com/paper/2009.10019/) (2020)
- [Feedback MPC for Torque-Controlled Legged Robots](https://mlsanity.com/paper/1905.06144/) (2019)
- [Learning Task Space Actions for Bipedal Locomotion](https://mlsanity.com/paper/2011.04741/) (2020)
- [From Pixels to Legs: Hierarchical Learning of Quadruped Locomotion](https://mlsanity.com/paper/2011.11722/) (2020)
- [Learning Generalizable Locomotion Skills with Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1909.12324/) (2019)
- [Real-Time Motion Planning of Legged Robots: A Model Predictive Control Approach](https://mlsanity.com/paper/1710.04029/) (2017)
- [Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Biped](https://mlsanity.com/paper/1809.10811/) (2018)
- [Passive Whole-body Control for Quadruped Robots: Experimental Validation over Challenging Terrain](https://mlsanity.com/paper/1811.00884/) (2018)

## Built on by (most cited first, 30 of 54)

- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior](https://mlsanity.com/paper/2212.03238/) (2022)
- [Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions](https://mlsanity.com/paper/2203.15103/) (2022)
- [A Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning](https://mlsanity.com/paper/2208.07860/) (2022)
- [Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers](https://mlsanity.com/paper/2107.03996/) (2021)
- [Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots](https://mlsanity.com/paper/2111.01674/) (2021)
- [Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer](https://mlsanity.com/paper/2404.05695/) (2024)
- [Advanced Skills by Learning Locomotion and Local Navigation End-to-End](https://mlsanity.com/paper/2209.12827/) (2022)
- [Coupling Vision and Proprioception for Navigation of Legged Robots](https://mlsanity.com/paper/2112.02094/) (2021)
- [Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robots](https://mlsanity.com/paper/2405.01792/) (2024)
- [Learning to Jump from Pixels](https://mlsanity.com/paper/2110.15344/) (2021)
- [Prompt a Robot to Walk with Large Language Models](https://mlsanity.com/paper/2309.09969/) (2023)
- [Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion](https://mlsanity.com/paper/2303.11330/) (2023)
- [Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robot](https://mlsanity.com/paper/2208.01160/) (2022)
- [CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion](https://mlsanity.com/paper/2211.00458/) (2022)
- [Learning Quadruped Locomotion Using Differentiable Simulation](https://mlsanity.com/paper/2403.14864/) (2024)
- [Learning Torque Control for Quadrupedal Locomotion](https://mlsanity.com/paper/2203.05194/) (2022)
- [Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion](https://mlsanity.com/paper/2310.17634/) (2023)
- [ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robots](https://mlsanity.com/paper/2310.10486/) (2023)
- [Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomization](https://mlsanity.com/paper/2209.12878/) (2022)
- [Sim-to-Real Transfer for Quadrupedal Locomotion via Terrain Transformer](https://mlsanity.com/paper/2212.07740/) (2022)
- [Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing](https://mlsanity.com/paper/2411.07104/) (2024)
- [Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion](https://mlsanity.com/paper/2209.14887/) (2022)
- [Learning to Walk by Steering: Perceptive Quadrupedal Locomotion in Dynamic Environments](https://mlsanity.com/paper/2209.09233/) (2022)
- [Continuous Versatile Jumping Using Learned Action Residuals](https://mlsanity.com/paper/2304.08663/) (2023)
- [Gaitor: Learning a Unified Representation Across Gaits for Real-World Quadruped Locomotion](https://mlsanity.com/paper/2405.19452/) (2024)
- [Learning Quadruped Locomotion Policies using Logical Rules](https://mlsanity.com/paper/2107.10969/) (2021)
- [Learning Semantics-Aware Locomotion Skills from Human Demonstration](https://mlsanity.com/paper/2206.13631/) (2022)
- [Puppeteer and Marionette: Learning Anticipatory Quadrupedal Locomotion Based on Interactions of a Central Pattern Generator and Supraspinal Drive](https://mlsanity.com/paper/2302.13378/) (2023)

## Abstract (arXiv)

We focus on the problem of developing energy efficient controllers for quadrupedal robots. Animals can actively switch gaits at different speeds to lower their energy consumption. In this paper, we devise a hierarchical learning framework, in which distinctive locomotion gaits and natural gait transitions emerge automatically with a simple reward of energy minimization. We use evolutionary strategies (ES) to train a high-level gait policy that specifies gait patterns of each foot, while the low-level convex MPC controller optimizes the motor commands so that the robot can walk at a desired velocity using that gait pattern. We test our learning framework on a quadruped robot and demonstrate automatic gait transitions, from walking to trotting and to fly-trotting, as the robot increases its speed. We show that the learned hierarchical controller consumes much less energy across a wide range of locomotion speed than baseline controllers.