# Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning

arXiv: https://arxiv.org/abs/2301.10737 · HTML: https://mlsanity.com/paper/2301.10737/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 25 Jan 2023.

## Builds on (most cited first, 12 of 12)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control](https://mlsanity.com/paper/1611.03537/) (2016)
- [Dynamic mode decomposition with control](https://mlsanity.com/paper/1409.6358/) (2014)
- [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.07169/) (2020)
- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Koopman operator-based model reduction for switched-system control of PDEs](https://mlsanity.com/paper/1710.06759/) (2017)
- [Control of chaotic systems by Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07672/) (2019)
- [Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control](https://mlsanity.com/paper/2108.00128/) (2021)
- [Data-Driven Model Predictive Control using Interpolated Koopman Generators](https://mlsanity.com/paper/2003.07094/) (2020)
- [Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics](https://mlsanity.com/paper/2104.05437/) (2021)
- [Explicit multiobjective model predictive control for nonlinear systems with symmetries](https://mlsanity.com/paper/1809.06238/) (2018)

## Built on by (most cited first, 24 of 24)

- [Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies](https://mlsanity.com/paper/2403.15267/) (2024)
- [Learning a model is paramount for sample efficiency in reinforcement learning control of PDEs](https://mlsanity.com/paper/2302.07160/) (2023)
- [Equivariance and partial observations in Koopman operator theory for partial differential equations](https://mlsanity.com/paper/2307.15325/) (2023)
- [Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms](https://mlsanity.com/paper/2311.18736/) (2023)
- [Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art](https://mlsanity.com/paper/2412.01566/) (2024)
- [Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and quality](https://mlsanity.com/paper/2407.17822/) (2024)
- [Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models](https://mlsanity.com/paper/2409.05709/) (2024)
- [HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systems](https://mlsanity.com/paper/2501.04538/) (2025)
- [Surrogate Modeling of 3D Rayleigh-Benard Convection with Equivariant Autoencoders](https://mlsanity.com/paper/2505.13569/) (2025)
- [Solving Partial Differential Equations with Equivariant Extreme Learning Machines](https://mlsanity.com/paper/2404.18530/) (2024)
- [Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems](https://mlsanity.com/paper/2411.04098/) (2024)
- [Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control](https://mlsanity.com/paper/2601.15015/) (2026)
- [Reinforcement Learning for Freeway Lane-Change Regulation via Connected Vehicles](https://mlsanity.com/paper/2412.04341/) (2024)
- [Physics-Informed Neural Networks with Unknown Partial Differential Equations: an Application in Multivariate Time Series](https://mlsanity.com/paper/2503.20144/) (2025)
- [Invariant Control Strategies for Active Flow Control using Graph Neural Networks](https://mlsanity.com/paper/2503.22775/) (2025)
- [HypeMARL: Multi-Agent Reinforcement Learning For High-Dimensional, Parametric, and Distributed Systems](https://mlsanity.com/paper/2509.16709/) (2025)
- [Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models](https://mlsanity.com/paper/2412.09942/) (2024)
- [Neural Operator based Reinforcement Learning for Control of first-order PDEs with Spatially-Varying State Delay](https://mlsanity.com/paper/2501.18201/) (2025)
- [Transfer learning strategies for accelerating reinforcement-learning-based flow control](https://mlsanity.com/paper/2510.16016/) (2025)
- [Koopman-based surrogate modeling for reinforcement-learning-control of Rayleigh-Benard convection](https://mlsanity.com/paper/2603.28074/) (2026)
- [Hyperfastrl: Hypernetwork-based reinforcement learning for unified control of parametric chaotic PDEs](https://mlsanity.com/paper/2604.06497/) (2026)
- [Physics-enhanced reinforcement learning for real-time optimal control of dynamical systems](https://mlsanity.com/paper/2607.16177/) (2026)
- [HypEMBER: Hypernetwork-based Ensemble for Robust Policy Learning of Parametrized Dynamical Systems](https://mlsanity.com/paper/2607.19628/) (2026)
- [Backstepping-Guided Reinforcement Learning for Wide-Range Saint-Venant Canal Regulation](https://mlsanity.com/paper/2608.20089/) (2026)

## Abstract (arXiv)

We present a convolutional framework which significantly reduces the complexity and thus, the computational effort for distributed reinforcement learning control of dynamical systems governed by partial differential equations (PDEs). Exploiting translational invariances, the high-dimensional distributed control problem can be transformed into a multi-agent control problem with many identical, uncoupled agents. Furthermore, using the fact that information is transported with finite velocity in many cases, the dimension of the agents' environment can be drastically reduced using a convolution operation over the state space of the PDE. In this setting, the complexity can be flexibly adjusted via the kernel width or by using a stride greater than one. Moreover, scaling from smaller to larger systems -- or the transfer between different domains -- becomes a straightforward task requiring little effort. We demonstrate the performance of the proposed framework using several PDE examples with increasing complexity, where stabilization is achieved by training a low-dimensional deep deterministic policy gradient agent using minimal computing resources.