# Quadruped Locomotion on Non-Rigid Terrain using Reinforcement Learning

arXiv: https://arxiv.org/abs/2107.02955 · HTML: https://mlsanity.com/paper/2107.02955/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 7 Jul 2021.

## Builds on (most cited first, 26 of 26)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Sim-to-Real: Learning Agile Locomotion For Quadruped Robots](https://mlsanity.com/paper/1804.10332/) (2018)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [Emergence of Locomotion Behaviours in Rich Environments](https://mlsanity.com/paper/1707.02286/) (2017)
- [DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills](https://mlsanity.com/paper/1804.02717/) (2018)
- [Learning to Walk via Deep Reinforcement Learning](https://mlsanity.com/paper/1812.11103/) (2018)
- [Learning to Walk in the Real World with Minimal Human Effort](https://mlsanity.com/paper/2002.08550/) (2020)
- [Data Efficient Reinforcement Learning for Legged Robots](https://mlsanity.com/paper/1907.03613/) (2019)
- [Learning Symmetric and Low-energy Locomotion](https://mlsanity.com/paper/1801.08093/) (2018)
- [ALLSTEPS: Curriculum-driven Learning of Stepping Stone Skills](https://mlsanity.com/paper/2005.04323/) (2020)
- [DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learning](https://mlsanity.com/paper/1909.08399/) (2019)
- [Feedback Control For Cassie With Deep Reinforcement Learning](https://mlsanity.com/paper/1803.05580/) (2018)
- [Robust Recovery Controller for a Quadrupedal Robot using Deep Reinforcement Learning](https://mlsanity.com/paper/1901.07517/) (2019)
- [Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion](https://mlsanity.com/paper/2002.09676/) (2020)
- [Whole-Body Nonlinear Model Predictive Control Through Contacts for Quadrupeds](https://mlsanity.com/paper/1712.02889/) (2017)
- [Circus ANYmal: A Quadruped Learning Dexterous Manipulation with Its Limbs](https://mlsanity.com/paper/2011.08811/) (2020)
- [Training in Task Space to Speed Up and Guide Reinforcement Learning](https://mlsanity.com/paper/1903.02219/) (2019)
- [CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion](https://mlsanity.com/paper/2005.03288/) (2020)
- [STANCE: Locomotion Adaptation over Soft Terrain](https://mlsanity.com/paper/1904.12306/) (2019)
- [Dynamic Legged Manipulation of a Ball Through Multi-Contact Optimization](https://mlsanity.com/paper/2008.00191/) (2020)
- [Contact-Implicit Trajectory Optimization using an Analytically Solvable Contact Model for Locomotion on Variable Ground](https://mlsanity.com/paper/2007.11261/) (2020)
- [Dynamics and Domain Randomized Gait Modulation with Bezier Curves for Sim-to-Real Legged Locomotion](https://mlsanity.com/paper/2010.12070/) (2020)

## Built on by (most cited first, 2 of 2)

- [Efficient Reinforcement Learning for Jumping Monopods](https://mlsanity.com/paper/2309.07038/) (2023)
- [SATA: Safe and Adaptive Torque-Based Locomotion Policies Inspired by Animal Learning](https://mlsanity.com/paper/2502.12674/) (2025)

## Abstract (arXiv)

Legged robots need to be capable of walking on diverse terrain conditions. In this paper, we present a novel reinforcement learning framework for learning locomotion on non-rigid dynamic terrains. Specifically, our framework can generate quadruped locomotion on flat elastic terrain that consists of a matrix of tiles moving up and down passively when pushed by the robot's feet. A trained robot with 55cm base length can walk on terrain that can sink up to 5cm. We propose a set of observation and reward terms that enable this locomotion; in which we found that it is crucial to include the end-effector history and end-effector velocity terms into observation. We show the effectiveness of our method by training the robot with various terrain conditions.