# Leveraging Symmetry in RL-based Legged Locomotion Control

arXiv: https://arxiv.org/abs/2403.17320 · HTML: https://mlsanity.com/paper/2403.17320/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 31 of its 38 references that are in the map.
- First posted on arXiv 26 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 30 of 31)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [Learning Agile Robotic Locomotion Skills by Imitating Animals](https://mlsanity.com/paper/2004.00784/) (2020)
- [Efficient Exploration via State Marginal Matching](https://mlsanity.com/paper/1906.05274/) (2019)
- [MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning](https://mlsanity.com/paper/2006.16908/) (2020)
- [End-to-End Robotic Reinforcement Learning without Reward Engineering](https://mlsanity.com/paper/1904.07854/) (2019)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [SO(2)-Equivariant Reinforcement Learning](https://mlsanity.com/paper/2203.04439/) (2022)
- [Learning Symmetric and Low-energy Locomotion](https://mlsanity.com/paper/1801.08093/) (2018)
- [Equivariant Transporter Network](https://mlsanity.com/paper/2202.09400/) (2022)
- [Creative Robot Tool Use with Large Language Models](https://mlsanity.com/paper/2310.13065/) (2023)
- [On-Robot Learning With Equivariant Models](https://mlsanity.com/paper/2203.04923/) (2022)
- [DribbleBot: Dynamic Legged Manipulation in the Wild](https://mlsanity.com/paper/2304.01159/) (2023)
- [Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion](https://mlsanity.com/paper/2303.11330/) (2023)
- [Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning](https://mlsanity.com/paper/2203.14912/) (2022)
- [Robust and Versatile Bipedal Jumping Control through Reinforcement Learning](https://mlsanity.com/paper/2302.09450/) (2023)
- [CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion](https://mlsanity.com/paper/2211.00458/) (2022)
- [Learning and Adapting Agile Locomotion Skills by Transferring Experience](https://mlsanity.com/paper/2304.09834/) (2023)
- [Symmetry Considerations for Learning Task Symmetric Robot Policies](https://mlsanity.com/paper/2403.04359/) (2024)
- [Continuous MDP Homomorphisms and Homomorphic Policy Gradient](https://mlsanity.com/paper/2209.07364/) (2022)
- [Group Equivariant Deep Reinforcement Learning](https://mlsanity.com/paper/2007.03437/) (2020)
- [Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learning](https://mlsanity.com/paper/2210.04435/) (2022)
- [Learning Agile Bipedal Motions on a Quadrupedal Robot](https://mlsanity.com/paper/2311.05818/) (2023)
- [Morphological Symmetries in Robotics](https://mlsanity.com/paper/2402.15552/) (2024)
- [On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis](https://mlsanity.com/paper/2302.10433/) (2023)
- [A CPG-Based Agile and Versatile Locomotion Framework Using Proximal Symmetry Loss](https://mlsanity.com/paper/2103.00928/) (2021)
- [An Adaptable Approach to Learn Realistic Legged Locomotion without Examples](https://mlsanity.com/paper/2110.14998/) (2021)
- [Multi-Modal Legged Locomotion Framework with Automated Residual Reinforcement Learning](https://mlsanity.com/paper/2202.12033/) (2022)

## Built on by (most cited first, 25 of 25)

- [HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit](https://mlsanity.com/paper/2502.13013/) (2025)
- [DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasets](https://mlsanity.com/paper/2404.19264/) (2024)
- [Learning Getting-Up Policies for Real-World Humanoid Robots](https://mlsanity.com/paper/2502.12152/) (2025)
- [MoE-Loco: Mixture of Experts for Multitask Locomotion](https://mlsanity.com/paper/2503.08564/) (2025)
- [Humanoid Whole-Body Locomotion on Narrow Terrain via Dynamic Balance and Reinforcement Learning](https://mlsanity.com/paper/2502.17219/) (2025)
- [Robust Robot Walker: Learning Agile Locomotion over Tiny Traps](https://mlsanity.com/paper/2409.07409/) (2024)
- [Morphological-Symmetry-Equivariant Heterogeneous Graph Neural Network for Robotic Dynamics Learning](https://mlsanity.com/paper/2412.01297/) (2024)
- [Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulation](https://mlsanity.com/paper/2505.05287/) (2025)
- [Guided Reinforcement Learning for Omnidirectional 3D Jumping in Quadruped Robots](https://mlsanity.com/paper/2507.16481/) (2025)
- [Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning](https://mlsanity.com/paper/2512.00727/) (2025)
- [ECHO: Edge-Cloud Humanoid Orchestration for Language-to-Motion Control](https://mlsanity.com/paper/2603.16188/) (2026)
- [Symmetry-Guided Memory Augmentation for Efficient Locomotion Learning](https://mlsanity.com/paper/2502.01521/) (2025)
- [Bipedalism for Quadrupedal Robots: Versatile Loco-Manipulation through Risk-Adaptive Reinforcement Learning](https://mlsanity.com/paper/2507.20382/) (2025)
- [QuietPaw: Learning Quadrupedal Locomotion with Versatile Noise Preference Alignment](https://mlsanity.com/paper/2503.05035/) (2025)
- [Learning to Walk With Less: A Dyna-Style Approach to Quadrupedal Locomotion](https://mlsanity.com/paper/2509.06296/) (2025)
- [Toward Reliable Sim-to-Real Predictability for MoE-based Robust Quadrupedal Locomotion](https://mlsanity.com/paper/2602.00678/) (2026)
- [Symmetries-enhanced Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2501.01136/) (2025)
- [ECO: Energy-Constrained Optimization with Reinforcement Learning for Humanoid Walking](https://mlsanity.com/paper/2602.06445/) (2026)
- [Scoop-and-Toss: Dynamic Object Collection for Quadrupedal Systems](https://mlsanity.com/paper/2506.09406/) (2025)
- [Multi-Task Reinforcement Learning of Drone Aerobatics by Exploiting Geometric Symmetries](https://mlsanity.com/paper/2602.10997/) (2026)
- [Symmetry-Aware Fusion of Vision and Tactile Sensing via Bilateral Force Priors for Robotic Manipulation](https://mlsanity.com/paper/2602.13689/) (2026)
- [Morphologically Equivariant Flow Matching for Bimanual Mobile Manipulation](https://mlsanity.com/paper/2605.12228/) (2026)
- [Reflex: Reinforcement Learning with Reflection Symmetry Exploitation in State-Based Continuous Control](https://mlsanity.com/paper/2605.23415/) (2026)
- [SKooP: Symmetric Koopman Predictions for Faster and More Generalizable Legged Robot Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2607.11624/) (2026)
- [OmniMimic: Dynamics-completed Motion Augmentation for Multi-style Omnidirectional Quadruped Locomotion](https://mlsanity.com/paper/2609.20566/) (2026)

## Abstract (arXiv)

Model-free reinforcement learning is a promising approach for autonomously solving challenging robotics control problems, but faces exploration difficulty without information of the robot's kinematics and dynamics morphology. The under-exploration of multiple modalities with symmetric states leads to behaviors that are often unnatural and sub-optimal. This issue becomes particularly pronounced in the context of robotic systems with morphological symmetries, such as legged robots for which the resulting asymmetric and aperiodic behaviors compromise performance, robustness, and transferability to real hardware. To mitigate this challenge, we can leverage symmetry to guide and improve the exploration in policy learning via equivariance/invariance constraints. In this paper, we investigate the efficacy of two approaches to incorporate symmetry: modifying the network architectures to be strictly equivariant/invariant, and leveraging data augmentation to approximate equivariant/invariant actor-critics. We implement the methods on challenging loco-manipulation and bipedal locomotion tasks and compare with an unconstrained baseline. We find that the strictly equivariant policy consistently outperforms other methods in sample efficiency and task performance in simulation. In addition, symmetry-incorporated approaches exhibit better gait quality, higher robustness and can be deployed zero-shot in real-world experiments.