# Symmetry Detection in Trajectory Data for More Meaningful Reinforcement Learning Representations

arXiv: https://arxiv.org/abs/2211.16381 · HTML: https://mlsanity.com/paper/2211.16381/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 18 references that are in the map.
- First posted on arXiv 29 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Explainable Reinforcement Learning Through a Causal Lens](https://mlsanity.com/paper/1905.10958/) (2019)
- [Explainable Reinforcement Learning: A Survey](https://mlsanity.com/paper/2005.06247/) (2020)
- [Explainability in Deep Reinforcement Learning](https://mlsanity.com/paper/2008.06693/) (2020)
- [Symmetry-Based Disentangled Representation Learning requires Interaction with Environments](https://mlsanity.com/paper/1904.00243/) (2019)
- [Efficient Saliency Maps for Explainable AI](https://mlsanity.com/paper/1911.11293/) (2019)
- [Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning](https://mlsanity.com/paper/2207.04703/) (2022)
- [Learning Linear Temporal Properties from Noisy Data: A MaxSAT Approach](https://mlsanity.com/paper/2104.15083/) (2021)
- [Explaining Conditions for Reinforcement Learning Behaviors from Real and Imagined Data](https://mlsanity.com/paper/2011.09004/) (2020)
- [Uncertainty-Aware Signal Temporal Logic Inference](https://mlsanity.com/paper/2105.11545/) (2021)

## Abstract (arXiv)

Knowledge of the symmetries of reinforcement learning (RL) systems can be used to create compressed and semantically meaningful representations of a low-level state space. We present a method of automatically detecting RL symmetries directly from raw trajectory data without requiring active control of the system. Our method generates candidate symmetries and trains a recurrent neural network (RNN) to discriminate between the original trajectories and the transformed trajectories for each candidate symmetry. The RNN discriminator's accuracy for each candidate reveals how symmetric the system is under that transformation. This information can be used to create high-level representations that are invariant to all symmetries on a dataset level and to communicate properties of the RL behavior to users. We show in experiments on two simulated RL use cases (a pusher robot and a UAV flying in wind) that our method can determine the symmetries underlying both the environment physics and the trained RL policy.