mlsanityData to 7 Oct 2026

Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning

arXiv 2301.10737 ↗First posted 25 Jan 2023cs.LG math.OCcited by 21

Not on the map. Why?

Abstract (arXiv)

We present a convolutional framework which significantly reduces the complexity and thus, the computational effort for distributed reinforcement learning control of dynamical systems governed by partial differential equations (PDEs). Exploiting translational invariances, the high-dimensional distributed control problem can be transformed into a multi-agent control problem with many identical, uncoupled agents. Furthermore, using the fact that information is transported with finite velocity in many cases, the dimension of the agents' environment can be drastically reduced using a convolution operation over the state space of the PDE. In this setting, the complexity can be flexibly adjusted via the kernel width or by using a stride greater than one. Moreover, scaling from smaller to larger systems -- or the transfer between different domains -- becomes a straightforward task requiring little effort. We demonstrate the performance of the proposed framework using several PDE examples with increasing complexity, where stabilization is achieved by training a low-dimensional deep deterministic policy gradient agent using minimal computing resources.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 12 of its 47 references that are in the map. First posted on arXiv 25 Jan 2023.

Builds on

12 of 47 references.

References · 12

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2019Machine Learning for Fluid Mechanicscited by 380
  3. 2016Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive controlcited by 134
  4. 2014Dynamic mode decomposition with controlcited by 128
  5. 2020Shared Experience Actor-Critic for Multi-Agent Reinforcement Learningcited by 114
  6. 2021Scientific multi-agent reinforcement learning for wall-models of turbulent flowscited by 52
  7. 2017Koopman operator-based model reduction for switched-system control of PDEscited by 31
  8. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 28
  9. 2021Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Controlcited by 22
  10. 2020Data-Driven Model Predictive Control using Interpolated Koopman Generatorscited by 17
  11. 2021Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamicscited by 10
  12. 2018Explicit multiobjective model predictive control for nonlinear systems with symmetriescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 7; 2026: 4.

21 papers.

Sort:

  1. 2024Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policiescited by 7
  2. 2023Equivariance and partial observations in Koopman operator theory for partial differential equationscited by 6
  3. 2024Multi-objective Deep Learning: Taxonomy and Survey of the State of the Artcited by 6
  4. 2023Learning a model is paramount for sample efficiency in reinforcement learning control of PDEscited by 5
  5. 2024Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and qualitycited by 5
  6. 2023Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithmscited by 4
  7. 2024Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order modelscited by 4
  8. 2025HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systemscited by 3
  9. 2025Surrogate Modeling of 3D Rayleigh-Benard Convection with Equivariant Autoencoderscited by 3
  10. 2024Solving Partial Differential Equations with Equivariant Extreme Learning Machinescited by 2
  11. 2024Interpretable and Efficient Data-driven Discovery and Control of Distributed Systemscited by 2
  12. 2026Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Controlcited by 2
9 more
  1. 2025Physics-Informed Neural Networks with Unknown Partial Differential Equations: an Application in Multivariate Time Seriescited by 1
  2. 2025Invariant Control Strategies for Active Flow Control using Graph Neural Networkscited by 1
  3. 2025HypeMARL: Multi-Agent Reinforcement Learning For High-Dimensional, Parametric, and Distributed Systemscited by 1
  4. 2024Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order modelscited by 0
  5. 2025Neural Operator based Reinforcement Learning for Control of first-order PDEs with Spatially-Varying State Delaycited by 0
  6. 2025Transfer learning strategies for accelerating reinforcement-learning-based flow controlcited by 0
  7. 2026Koopman-based surrogate modeling for reinforcement-learning-control of Rayleigh-Benard convectioncited by 0
  8. 2026Physics-enhanced reinforcement learning for real-time optimal control of dynamical systemscited by 0
  9. 2026HypEMBER: Hypernetwork-based Ensemble for Robust Policy Learning of Parametrized Dynamical Systemscited by 0