# Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control

arXiv: https://arxiv.org/abs/1806.06931 · HTML: https://mlsanity.com/paper/1806.06931/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2018.

## Built on by (most cited first, 11 of 11)

- [Wavelet Diffusion Neural Operator](https://mlsanity.com/paper/2412.04833/) (2024)
- [DiffPhyCon: A Generative Approach to Control Complex Physical Systems](https://mlsanity.com/paper/2407.06494/) (2024)
- [A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.03859/) (2022)
- [Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms](https://mlsanity.com/paper/2311.18736/) (2023)
- [Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations](https://mlsanity.com/paper/2110.11265/) (2021)
- [From Uncertain to Safe: Conformal Adaptation of Diffusion Models for Safe PDE Control](https://mlsanity.com/paper/2502.02205/) (2025)
- [Value Bonuses using Ensemble Errors for Exploration in Reinforcement Learning](https://mlsanity.com/paper/2602.12375/) (2026)
- [Precision Physical Activity Prescription via Reinforcement Learning for Functional Actions](https://mlsanity.com/paper/2605.19208/) (2026)
- [Taming Waves: A Physically-Interpretable Machine Learning Framework for Realizable Control of Wave Dynamics](https://mlsanity.com/paper/2312.09460/) (2023)
- [Suppressing Modulation Instability with Reinforcement Learning](https://mlsanity.com/paper/2404.04310/) (2024)
- [Backstepping-Guided Reinforcement Learning for Wide-Range Saint-Venant Canal Regulation](https://mlsanity.com/paper/2608.20089/) (2026)

## Abstract (arXiv)

Recent work has shown that reinforcement learning (RL) is a promising approach to control dynamical systems described by partial differential equations (PDE). This paper shows how to use RL to tackle more general PDE control problems that have continuous high-dimensional action spaces with spatial relationship among action dimensions. In particular, we propose the concept of action descriptors, which encode regularities among spatially-extended action dimensions and enable the agent to control high-dimensional action PDEs. We provide theoretical evidence suggesting that this approach can be more sample efficient compared to a conventional approach that treats each action dimension separately and does not explicitly exploit the spatial regularity of the action space. The action descriptor approach is then used within the deep deterministic policy gradient algorithm. Experiments on two PDE control problems, with up to 256-dimensional continuous actions, show the advantage of the proposed approach over the conventional one.