# Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics

arXiv: https://arxiv.org/abs/2104.05437 · HTML: https://mlsanity.com/paper/2104.05437/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 9 Apr 2021.

## Builds on (most cited first, 15 of 15)

- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [Universal Invariant and Equivariant Graph Neural Networks](https://mlsanity.com/paper/1905.04943/) (2019)
- [Equivariance Through Parameter-Sharing](https://mlsanity.com/paper/1702.08389/) (2017)
- [Exploiting Cyclic Symmetry in Convolutional Neural Networks](https://mlsanity.com/paper/1602.02660/) (2016)
- [Rotation-invariant convolutional neural networks for galaxy morphology prediction](https://mlsanity.com/paper/1503.07077/) (2015)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [Flow Navigation by Smart Microswimmers via Reinforcement Learning](https://mlsanity.com/paper/1701.08848/) (2017)
- [Universal approximations of permutation invariant/equivariant functions by deep neural networks](https://mlsanity.com/paper/1903.01939/) (2019)
- [Control of chaotic systems by Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07672/) (2019)
- [Data-Driven Stabilization of Periodic Orbits](https://mlsanity.com/paper/2010.13896/) (2020)
- [Restoring Chaos Using Deep Reinforcement Learning](https://mlsanity.com/paper/1912.00947/) (2019)
- [A statistical learning strategy for closed-loop control of fluid flows](https://mlsanity.com/paper/1604.03392/) (2016)

## Built on by (most cited first, 11 of 11)

- [Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning](https://mlsanity.com/paper/2301.10737/) (2023)
- [Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning](https://mlsanity.com/paper/2205.00579/) (2022)
- [Learning a model is paramount for sample efficiency in reinforcement learning control of PDEs](https://mlsanity.com/paper/2302.07160/) (2023)
- [Addressing imperfect symmetry: A novel symmetry-learning actor-critic extension](https://mlsanity.com/paper/2309.02711/) (2023)
- [Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and quality](https://mlsanity.com/paper/2407.17822/) (2024)
- [Building symmetries into data-driven manifold dynamics models for complex flows: application to two-dimensional Kolmogorov flow](https://mlsanity.com/paper/2312.10235/) (2023)
- [Solving Partial Differential Equations with Equivariant Extreme Learning Machines](https://mlsanity.com/paper/2404.18530/) (2024)
- [Data-assimilated model-informed reinforcement learning](https://mlsanity.com/paper/2506.01755/) (2025)
- [Blending data and physics for reduced-order modeling of systems with spatiotemporal chaotic dynamics](https://mlsanity.com/paper/2507.21299/) (2025)
- [Transfer learning strategies for accelerating reinforcement-learning-based flow control](https://mlsanity.com/paper/2510.16016/) (2025)
- [Hyperfastrl: Hypernetwork-based reinforcement learning for unified control of parametric chaotic PDEs](https://mlsanity.com/paper/2604.06497/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (RL) is a data-driven, model-free method capable of discovering complex control strategies for macroscopic objectives in high-dimensional systems, making its application towards flow control promising. Many systems of flow control interest possess symmetries that, when neglected, can significantly inhibit the learning and performance of a naive deep RL approach. Using a test-bed consisting of the Kuramoto-Sivashinsky Equation (KSE), equally spaced actuators, and a goal of minimizing dissipation and power cost, we demonstrate that by moving the deep RL problem to a symmetry-reduced space, we can alleviate limitations inherent in the naive application of deep RL. We demonstrate that symmetry-reduced deep RL yields improved data efficiency as well as improved control policy efficacy compared to policies found by naive deep RL. Interestingly, the policy learned by the the symmetry aware control agent drives the system toward an equilibrium state of the forced KSE that is connected by continuation to an equilibrium of the unforced KSE, despite having been given no explicit information regarding its existence. I.e., to achieve its goal, the RL algorithm discovers and stabilizes an equilibrium state of the system. Finally, we demonstrate that the symmetry-reduced control policy is robust to observation and actuation signal noise, as well as to system parameters it has not observed before.