mlsanityData to 14 Oct 2026

PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equations

arXiv 2405.11401 ↗First posted 18 May 2024eess.SY cs.AI cs.CEcited by 10

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Over the last decade, data-driven methods have surged in popularity, emerging as valuable tools for control theory. As such, neural network approximations of control feedback laws, system dynamics, and even Lyapunov functions have attracted growing attention. With the ascent of learning based control, the need for accurate, fast, and easy-to-use benchmarks has increased. In this work, we present the first learning-based environment for boundary control of PDEs. In our benchmark, we introduce three foundational PDE problems - a 1D transport PDE, a 1D reaction-diffusion PDE, and a 2D Navier-Stokes PDE - whose solvers are bundled in an user-friendly reinforcement learning gym. With this gym, we then present the first set of model-free, reinforcement learning algorithms for solving this series of benchmark problems, achieving stability, although at a higher cost compared to model-based PDE backstepping. With the set of benchmark environments and detailed examples, this work significantly lowers the barrier to entry for learning-based PDE control - a topic largely unexplored by the data-driven control community. The entire benchmark is available on Github along with detailed documentation and the presented reinforcement learning models are open sourced.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 46 references that are in the map. First posted on arXiv 18 May 2024.

Builds on

21 of 46 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  4. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  5. 2018DeepMind Control Suitecited by 956
  6. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  7. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 413
  8. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  9. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  10. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 197
  11. 2020Learning to Control PDEs with Differentiable Physicscited by 132
  12. 2018Lipschitz Continuity in Model-based Reinforcement Learningcited by 91
9 more
  1. 2021Optimal control of PDEs using physics-informed neural networkscited by 52
  2. 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 50
  3. 2023Neural Operators for Bypassing Gain and Control Computations in PDE Backsteppingcited by 29
  4. 2023Neural Operators of Backstepping Controller and Observer Gain Functions for Reaction-Diffusion PDEscited by 18
  5. 2020Combining Prior Knowledge and Data for Robust Controller Designcited by 9
  6. 2022Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Controlcited by 9
  7. 2022Machine Learning Accelerated PDE Backstepping Observerscited by 7
  8. 2024Ventilation and Temperature Control for Energy-efficient and Healthy Buildings: A Differentiable PDE Approachcited by 3
  9. 2013Modeling Information Diffusion in Online Social Networks with Partial Differential Equationscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 1; 2026: 5.

10 papers.

  1. 2024Adaptive control of reaction-diffusion PDEs via neural operator-approximated gain kernelscited by 5
  2. 2026ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environmentscited by 4
  3. 2024Interpretable and Efficient Data-driven Discovery and Control of Distributed Systemscited by 2
  4. 2026Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Controlcited by 2
  5. 2026PDEInvBench: A Comprehensive Dataset and Design Space Exploration of Neural Networks for PDE Inverse Problemscited by 2
  6. 2024ConDiff: A Challenging Dataset for Neural Solvers of Partial Differential Equationscited by 1
  7. 2024Safe PDE Boundary Control with Neural Operatorscited by 1
  8. 2025Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEscited by 0
  9. 2026Toward Goal-Agnostic Joint-Embedding Predictive Control of Partial Differential Equationscited by 0
  10. 2026Backstepping-Guided Reinforcement Learning for Wide-Range Saint-Venant Canal Regulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Policy Optimization for PDE Control with a Warm Start
  2. 2024Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies
  3. 2023PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
  4. 2024Text2PDE: Latent Diffusion Models for Accessible Physics Simulation
  5. 2026Learning to Discretize: Diffusion-Based Adaptive Mesh with Spectral Guidance
  6. 2026AutoPDEBench: Benchmarking LLM Auto-Research for Neural PDE Solver Design
  7. 2026Bias-Constrained Diffusion Schedules for PDE Emulations: Reconstruction Error Minimization and Efficient Unrolled Training
  8. 2025From Uncertain to Safe: Conformal Adaptation of Diffusion Models for Safe PDE Control