# Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behavior

arXiv: https://arxiv.org/abs/2212.03238 · HTML: https://mlsanity.com/paper/2212.03238/

## Key facts

- Cited by 138 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 29 references that are in the map.
- First posted on arXiv 6 Dec 2022.

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [Robots that can adapt like animals](https://mlsanity.com/paper/1407.3501/) (2014)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition](https://mlsanity.com/paper/2011.01387/) (2020)
- [Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion](https://mlsanity.com/paper/2202.05481/) (2022)
- [One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RL](https://mlsanity.com/paper/2010.14484/) (2020)
- [Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots](https://mlsanity.com/paper/2111.01674/) (2021)
- [Highly Dynamic Quadruped Locomotion via Whole-Body Impulse Control and Model Predictive Control](https://mlsanity.com/paper/1909.06586/) (2019)
- [Fast and Efficient Locomotion via Learned Gait Transitions](https://mlsanity.com/paper/2104.04644/) (2021)
- [Learning to Jump from Pixels](https://mlsanity.com/paper/2110.15344/) (2021)
- [Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning](https://mlsanity.com/paper/2203.14912/) (2022)
- [Learning Free Gait Transition for Quadruped Robots via Phase-Guided Controller](https://mlsanity.com/paper/2201.00206/) (2022)
- [Redeeming Intrinsic Rewards via Constrained Optimization](https://mlsanity.com/paper/2211.07627/) (2022)
- [Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires](https://mlsanity.com/paper/2109.08522/) (2021)
- [Learning a subspace of policies for online adaptation in Reinforcement Learning](https://mlsanity.com/paper/2110.05169/) (2021)
- [Sim-to-Real Learning of Footstep-Constrained Bipedal Dynamic Walking](https://mlsanity.com/paper/2203.07589/) (2022)

## Built on by (most cited first, 30 of 138)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusion](https://mlsanity.com/paper/2508.08241/) (2025)
- [Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding](https://mlsanity.com/paper/2503.13377/) (2025)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [DrEureka: Language Model Guided Sim-To-Real Transfer](https://mlsanity.com/paper/2406.01967/) (2024)
- [HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit](https://mlsanity.com/paper/2502.13013/) (2025)
- [Visual Whole-Body Control for Legged Loco-Manipulation](https://mlsanity.com/paper/2403.16967/) (2024)
- [QUAR-VLA: Vision-Language-Action Model for Quadruped Robots](https://mlsanity.com/paper/2312.14457/) (2023)
- [Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control](https://mlsanity.com/paper/2412.07773/) (2024)
- [AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Control](https://mlsanity.com/paper/2505.03738/) (2025)
- [RSL-RL: A Learning Library for Robotics Research](https://mlsanity.com/paper/2509.10771/) (2025)
- [BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds](https://mlsanity.com/paper/2502.10363/) (2025)
- [One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotion](https://mlsanity.com/paper/2409.06366/) (2024)
- [DribbleBot: Dynamic Legged Manipulation in the Wild](https://mlsanity.com/paper/2304.01159/) (2023)
- [Dexterous Functional Grasping](https://mlsanity.com/paper/2312.02975/) (2023)
- [Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response](https://mlsanity.com/paper/2312.11460/) (2023)
- [Robust and Versatile Bipedal Jumping Control through Reinforcement Learning](https://mlsanity.com/paper/2302.09450/) (2023)
- [GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot](https://mlsanity.com/paper/2403.13358/) (2024)
- [MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models](https://mlsanity.com/paper/2503.08007/) (2025)
- [RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion](https://mlsanity.com/paper/2305.17842/) (2023)
- [Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimization](https://mlsanity.com/paper/2311.03351/) (2023)
- [Learning Force Control for Legged Manipulation](https://mlsanity.com/paper/2405.01402/) (2024)
- [LocoFormer: Generalist Locomotion via Long-context Adaptation](https://mlsanity.com/paper/2509.23745/) (2025)
- [AdaptSim: Task-Driven Simulation Adaptation for Sim-to-Real Transfer](https://mlsanity.com/paper/2302.04903/) (2023)
- [RobotKeyframing: Learning Locomotion with High-Level Objectives via Mixture of Dense and Sparse Rewards](https://mlsanity.com/paper/2407.11562/) (2024)
- [Towards Embodiment Scaling Laws in Robot Locomotion](https://mlsanity.com/paper/2505.05753/) (2025)
- [Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models](https://mlsanity.com/paper/2407.02666/) (2024)
- [Make-An-Agent: A Generalizable Policy Network Generator with Behavior-Prompted Diffusion](https://mlsanity.com/paper/2407.10973/) (2024)
- [SoftMimic: Learning Compliant Whole-body Control from Examples](https://mlsanity.com/paper/2510.17792/) (2025)

## Abstract (arXiv)

Learned locomotion policies can rapidly adapt to diverse environments similar to those experienced during training but lack a mechanism for fast tuning when they fail in an out-of-distribution test environment. This necessitates a slow and iterative cycle of reward and environment redesign to achieve good performance on a new task. As an alternative, we propose learning a single policy that encodes a structured family of locomotion strategies that solve training tasks in different ways, resulting in Multiplicity of Behavior (MoB). Different strategies generalize differently and can be chosen in real-time for new tasks or environments, bypassing the need for time-consuming retraining. We release a fast, robust open-source MoB locomotion controller, Walk These Ways, that can execute diverse gaits with variable footswing, posture, and speed, unlocking diverse downstream tasks: crouching, hopping, high-speed running, stair traversal, bracing against shoves, rhythmic dance, and more. Video and code release: https://gmargo11.github.io/walk-these-ways/