# Addressing imperfect symmetry: A novel symmetry-learning actor-critic extension

arXiv: https://arxiv.org/abs/2309.02711 · HTML: https://mlsanity.com/paper/2309.02711/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 50 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study](https://mlsanity.com/paper/2006.05990/) (2020)
- [MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning](https://mlsanity.com/paper/2006.16908/) (2020)
- [Equivariance Through Parameter-Sharing](https://mlsanity.com/paper/1702.08389/) (2017)
- [Learning Symmetric and Low-energy Locomotion](https://mlsanity.com/paper/1801.08093/) (2018)
- [Universal approximations of permutation invariant/equivariant functions by deep neural networks](https://mlsanity.com/paper/1903.01939/) (2019)
- [Group Equivariant Deep Reinforcement Learning](https://mlsanity.com/paper/2007.03437/) (2020)
- [Symmetry Learning for Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1706.02999/) (2017)
- [Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics](https://mlsanity.com/paper/2104.05437/) (2021)
- [A CPG-Based Agile and Versatile Locomotion Framework Using Proximal Symmetry Loss](https://mlsanity.com/paper/2103.00928/) (2021)
- [Augmenting learning using symmetry in a biologically-inspired domain](https://mlsanity.com/paper/1910.00528/) (2019)
- [Encoding Involutory Invariances in Neural Networks](https://mlsanity.com/paper/2106.12891/) (2021)

## Built on by (most cited first, 5 of 5)

- [Symmetry Considerations for Learning Task Symmetric Robot Policies](https://mlsanity.com/paper/2403.04359/) (2024)
- [Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning](https://mlsanity.com/paper/2510.03578/) (2025)
- [Coordinated Humanoid Robot Locomotion with Symmetry Equivariant Reinforcement Learning Policy](https://mlsanity.com/paper/2508.01247/) (2025)
- [Symmetric Linear Bandits with Hidden Symmetry](https://mlsanity.com/paper/2405.13899/) (2024)
- [Reflex: Reinforcement Learning with Reflection Symmetry Exploitation in State-Based Continuous Control](https://mlsanity.com/paper/2605.23415/) (2026)

## Abstract (arXiv)

Symmetry, a fundamental concept to understand our environment, often oversimplifies reality from a mathematical perspective. Humans are a prime example, deviating from perfect symmetry in terms of appearance and cognitive biases (e.g. having a dominant hand). Nevertheless, our brain can easily overcome these imperfections and efficiently adapt to symmetrical tasks. The driving motivation behind this work lies in capturing this ability through reinforcement learning. To this end, we introduce Adaptive Symmetry Learning (ASL), a model-minimization actor-critic extension that addresses incomplete or inexact symmetry descriptions by adapting itself during the learning process. ASL consists of a symmetry fitting component and a modular loss function that enforces a common symmetric relation across all states while adapting to the learned policy. The performance of ASL is compared to existing symmetry-enhanced methods in a case study involving a four-legged ant model for multidirectional locomotion tasks. The results show that ASL can recover from large perturbations and generalize knowledge to hidden symmetric states. It achieves comparable or better performance than alternative methods in most scenarios, making it a valuable approach for leveraging model symmetry while compensating for inherent perturbations.