# Leveraging Symmetry to Accelerate Learning of Trajectory Tracking Controllers for Free-Flying Robotic Systems

arXiv: https://arxiv.org/abs/2409.11238 · HTML: https://mlsanity.com/paper/2409.11238/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 17 of 17)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning](https://mlsanity.com/paper/2006.16908/) (2020)
- [Control of a Quadrotor with Reinforcement Learning](https://mlsanity.com/paper/1707.05110/) (2017)
- [Equivariant Q Learning in Spatial Action Spaces](https://mlsanity.com/paper/2110.15443/) (2021)
- [Sim-to-(Multi)-Real: Transfer of Low-Level Robust Control Policies to Multiple Quadrotors](https://mlsanity.com/paper/1903.04628/) (2019)
- [Continuous MDP Homomorphisms and Homomorphic Policy Gradient](https://mlsanity.com/paper/2209.07364/) (2022)
- [DATT: Deep Adaptive Trajectory Tracking for Quadrotor Control](https://mlsanity.com/paper/2310.09053/) (2023)
- [TinyMPC: Model-Predictive Control on Resource-Constrained Microcontrollers](https://mlsanity.com/paper/2310.16985/) (2023)
- [Policy Gradient Methods in the Presence of Symmetries and State Abstractions](https://mlsanity.com/paper/2305.05666/) (2023)
- [On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis](https://mlsanity.com/paper/2302.10433/) (2023)
- [Equivariant Reinforcement Learning for Quadrotor UAV](https://mlsanity.com/paper/2206.01233/) (2022)
- [An Error-State Model Predictive Control on Connected Matrix Lie Groups for Legged Robot Control](https://mlsanity.com/paper/2203.08728/) (2022)
- [Exploiting Different Symmetries for Trajectory Tracking Control with Application to Quadrotors](https://mlsanity.com/paper/2207.04782/) (2022)
- [The Geometry of Optimal Gaits for Inertia-dominated Kinematic Systems](https://mlsanity.com/paper/2102.01506/) (2021)
- [The Role of Symmetry in Constructing Geometric Flat Outputs for Free-Flying Robotic Systems](https://mlsanity.com/paper/2209.11869/) (2022)
- [Almost Global Asymptotic Trajectory Tracking for Fully-Actuated Mechanical Systems on Homogeneous Riemannian Manifolds](https://mlsanity.com/paper/2403.04900/) (2024)

## Built on by (most cited first, 2 of 2)

- [Leveling the Playing Field: Carefully Comparing Classical and Learned Controllers for Quadrotor Trajectory Tracking](https://mlsanity.com/paper/2506.17832/) (2025)
- [Zero-Shot Policy Transfer in Reinforcement Learning using Buckingham's Pi Theorem](https://mlsanity.com/paper/2510.08768/) (2025)

## Abstract (arXiv)

Tracking controllers enable robotic systems to accurately follow planned reference trajectories. In particular, reinforcement learning (RL) has shown promise in the synthesis of controllers for systems with complex dynamics and modest online compute budgets. However, the poor sample efficiency of RL and the challenges of reward design make training slow and sometimes unstable, especially for high-dimensional systems. In this work, we leverage the inherent Lie group symmetries of robotic systems with a floating base to mitigate these challenges when learning tracking controllers. We model a general tracking problem as a Markov decision process (MDP) that captures the evolution of both the physical and reference states. Next, we prove that symmetry in the underlying dynamics and running costs leads to an MDP homomorphism, a mapping that allows a policy trained on a lower-dimensional "quotient" MDP to be lifted to an optimal tracking controller for the original system. We compare this symmetry-informed approach to an unstructured baseline, using Proximal Policy Optimization (PPO) to learn tracking controllers for three systems: the Particle (a forced point mass), the Astrobee (a fullyactuated space robot), and the Quadrotor (an underactuated system). Results show that a symmetry-aware approach both accelerates training and reduces tracking error at convergence.