# PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equations

arXiv: https://arxiv.org/abs/2405.11401 · HTML: https://mlsanity.com/paper/2405.11401/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 18 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Towards Multi-spatiotemporal-scale Generalized PDE Modeling](https://mlsanity.com/paper/2209.15616/) (2022)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Lipschitz Continuity in Model-based Reinforcement Learning](https://mlsanity.com/paper/1804.07193/) (2018)
- [Optimal control of PDEs using physics-informed neural networks](https://mlsanity.com/paper/2111.09880/) (2021)
- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [Neural Operators for Bypassing Gain and Control Computations in PDE Backstepping](https://mlsanity.com/paper/2302.14265/) (2023)
- [Neural Operators of Backstepping Controller and Observer Gain Functions for Reaction-Diffusion PDEs](https://mlsanity.com/paper/2303.10506/) (2023)
- [Combining Prior Knowledge and Data for Robust Controller Design](https://mlsanity.com/paper/2009.05253/) (2020)
- [Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Control](https://mlsanity.com/paper/2209.07669/) (2022)
- [Machine Learning Accelerated PDE Backstepping Observers](https://mlsanity.com/paper/2211.15044/) (2022)
- [Ventilation and Temperature Control for Energy-efficient and Healthy Buildings: A Differentiable PDE Approach](https://mlsanity.com/paper/2403.08996/) (2024)
- [Modeling Information Diffusion in Online Social Networks with Partial Differential Equations](https://mlsanity.com/paper/1310.0505/) (2013)

## Built on by (most cited first, 10 of 10)

- [Adaptive control of reaction-diffusion PDEs via neural operator-approximated gain kernels](https://mlsanity.com/paper/2407.01745/) (2024)
- [ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environments](https://mlsanity.com/paper/2609.19134/) (2026)
- [Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems](https://mlsanity.com/paper/2411.04098/) (2024)
- [Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control](https://mlsanity.com/paper/2601.15015/) (2026)
- [PDEInvBench: A Comprehensive Dataset and Design Space Exploration of Neural Networks for PDE Inverse Problems](https://mlsanity.com/paper/2605.25353/) (2026)
- [ConDiff: A Challenging Dataset for Neural Solvers of Partial Differential Equations](https://mlsanity.com/paper/2406.04709/) (2024)
- [Safe PDE Boundary Control with Neural Operators](https://mlsanity.com/paper/2411.15643/) (2024)
- [Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs](https://mlsanity.com/paper/2507.04232/) (2025)
- [Toward Goal-Agnostic Joint-Embedding Predictive Control of Partial Differential Equations](https://mlsanity.com/paper/2607.21644/) (2026)
- [Backstepping-Guided Reinforcement Learning for Wide-Range Saint-Venant Canal Regulation](https://mlsanity.com/paper/2608.20089/) (2026)

## Abstract (arXiv)

Over the last decade, data-driven methods have surged in popularity, emerging as valuable tools for control theory. As such, neural network approximations of control feedback laws, system dynamics, and even Lyapunov functions have attracted growing attention. With the ascent of learning based control, the need for accurate, fast, and easy-to-use benchmarks has increased. In this work, we present the first learning-based environment for boundary control of PDEs. In our benchmark, we introduce three foundational PDE problems - a 1D transport PDE, a 1D reaction-diffusion PDE, and a 2D Navier-Stokes PDE - whose solvers are bundled in an user-friendly reinforcement learning gym. With this gym, we then present the first set of model-free, reinforcement learning algorithms for solving this series of benchmark problems, achieving stability, although at a higher cost compared to model-based PDE backstepping. With the set of benchmark environments and detailed examples, this work significantly lowers the barrier to entry for learning-based PDE control - a topic largely unexplored by the data-driven control community. The entire benchmark is available on Github along with detailed documentation and the presented reinforcement learning models are open sourced.