# Adapting Rapid Motor Adaptation for Bipedal Robots

arXiv: https://arxiv.org/abs/2205.15299 · HTML: https://mlsanity.com/paper/2205.15299/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 30 May 2022.

## Builds on (most cited first, 30 of 34)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learning](https://mlsanity.com/paper/1803.11347/) (2018)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [Learning to Walk via Deep Reinforcement Learning](https://mlsanity.com/paper/1812.11103/) (2018)
- [Preparing for the Unknown: Learning a Universal Policy with Online System Identification](https://mlsanity.com/paper/1702.02453/) (2017)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition](https://mlsanity.com/paper/2011.01387/) (2020)
- [Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2105.08328/) (2021)
- [Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots](https://mlsanity.com/paper/2111.01674/) (2021)
- [Sim-to-Real Transfer for Biped Locomotion](https://mlsanity.com/paper/1903.01390/) (2019)
- [Policies Modulating Trajectory Generators](https://mlsanity.com/paper/1910.02812/) (2019)
- [Learning Fast Adaptation with Meta Strategy Optimization](https://mlsanity.com/paper/1909.12995/) (2019)
- [Feedback Control For Cassie With Deep Reinforcement Learning](https://mlsanity.com/paper/1803.05580/) (2018)
- [Policy Transfer with Strategy Optimization](https://mlsanity.com/paper/1810.05751/) (2018)
- [Rapidly Adaptable Legged Robots via Evolutionary Meta-Learning](https://mlsanity.com/paper/2003.01239/) (2020)
- [Environment Probing Interaction Policies](https://mlsanity.com/paper/1907.11740/) (2019)
- [Learning Memory-Based Control for Human-Scale Bipedal Locomotion](https://mlsanity.com/paper/2006.02402/) (2020)
- [Feedback Control of a Cassie Bipedal Robot: Walking, Standing, and Riding a Segway](https://mlsanity.com/paper/1809.07279/) (2018)
- [Animated Cassie: A Dynamic Relatable Robotic Character](https://mlsanity.com/paper/2009.02846/) (2020)
- [Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning](https://mlsanity.com/paper/1910.01748/) (2019)
- [Rapid Trajectory Optimization Using C-FROST with Illustration on a Cassie-Series Dynamic Walking Biped](https://mlsanity.com/paper/1807.06614/) (2018)
- [An Efficiently Solvable Quadratic Program for Stabilizing Dynamic Locomotion](https://mlsanity.com/paper/1311.1839/) (2013)
- [Autonomous Navigation of Underactuated Bipedal Robots in Height-Constrained Environments](https://mlsanity.com/paper/2109.05714/) (2021)

## Built on by (most cited first, 30 of 54)

- [HumanPlus: Humanoid Shadowing and Imitation from Humans](https://mlsanity.com/paper/2406.10454/) (2024)
- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills](https://mlsanity.com/paper/2502.01143/) (2025)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting](https://mlsanity.com/paper/2409.10161/) (2024)
- [Visual Whole-Body Control for Legged Loco-Manipulation](https://mlsanity.com/paper/2403.16967/) (2024)
- [Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning](https://mlsanity.com/paper/2408.14472/) (2024)
- [Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer](https://mlsanity.com/paper/2404.05695/) (2024)
- [One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotion](https://mlsanity.com/paper/2409.06366/) (2024)
- [Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion](https://mlsanity.com/paper/2303.11330/) (2023)
- [Robust and Versatile Bipedal Jumping Control through Reinforcement Learning](https://mlsanity.com/paper/2302.09450/) (2023)
- [Learning Smooth Humanoid Locomotion through Lipschitz-Constrained Policies](https://mlsanity.com/paper/2410.11825/) (2024)
- [Learning Bipedal Walking for Humanoids with Current Feedback](https://mlsanity.com/paper/2303.03724/) (2023)
- [Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges](https://mlsanity.com/paper/2311.18044/) (2023)
- [Towards Embodiment Scaling Laws in Robot Locomotion](https://mlsanity.com/paper/2505.05753/) (2025)
- [DreamControl: Human-Inspired Whole-Body Humanoid Control for Scene Interaction via Guided Diffusion](https://mlsanity.com/paper/2509.14353/) (2025)
- [Learning Humanoid Locomotion with World Model Reconstruction](https://mlsanity.com/paper/2502.16230/) (2025)
- [Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identifications](https://mlsanity.com/paper/2410.20357/) (2024)
- [Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild](https://mlsanity.com/paper/2304.10888/) (2023)
- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [Robot Trains Robot: Automatic Real-World Policy Adaptation and Learning for Humanoids](https://mlsanity.com/paper/2508.12252/) (2025)
- [BiRoDiff: Diffusion policies for bipedal robot locomotion on unseen terrains](https://mlsanity.com/paper/2407.05424/) (2024)
- [Efficient Imitation under Misspecification](https://mlsanity.com/paper/2503.13162/) (2025)
- [Following the Human Thread in Social Navigation](https://mlsanity.com/paper/2404.11327/) (2024)
- [Moving Forward by Moving Backward: Embedding Action Impact over Action Semantics](https://mlsanity.com/paper/2304.12289/) (2023)
- [Synaptic motor adaptation: A three-factor learning rule for adaptive robotic control in spiking neural networks](https://mlsanity.com/paper/2306.01906/) (2023)
- [What Teaches Robots to Walk, Teaches Them to Trade too -- Regime Adaptive Execution using Informed Data and LLMs](https://mlsanity.com/paper/2406.15508/) (2024)
- [Spectral Normalization for Lipschitz-Constrained Policies on Learning Humanoid Locomotion](https://mlsanity.com/paper/2504.08246/) (2025)
- [Action Flow Matching for Continual Robot Learning](https://mlsanity.com/paper/2504.18471/) (2025)
- [Booster Gym: An End-to-End Reinforcement Learning Framework for Humanoid Robot Locomotion](https://mlsanity.com/paper/2506.15132/) (2025)

## Abstract (arXiv)

Recent advances in legged locomotion have enabled quadrupeds to walk on challenging terrains. However, bipedal robots are inherently more unstable and hence it's harder to design walking controllers for them. In this work, we leverage recent advances in rapid adaptation for locomotion control, and extend them to work on bipedal robots. Similar to existing works, we start with a base policy which produces actions while taking as input an estimated extrinsics vector from an adaptation module. This extrinsics vector contains information about the environment and enables the walking controller to rapidly adapt online. However, the extrinsics estimator could be imperfect, which might lead to poor performance of the base policy which expects a perfect estimator. In this paper, we propose A-RMA (Adapting RMA), which additionally adapts the base policy for the imperfect extrinsics estimator by finetuning it using model-free RL. We demonstrate that A-RMA outperforms a number of RL-based baseline controllers and model-based controllers in simulation, and show zero-shot deployment of a single A-RMA policy to enable a bipedal robot, Cassie, to walk in a variety of different scenarios in the real world beyond what it has seen during training. Videos and results at https://ashish-kmr.github.io/a-rma/