mlsanityData to 14 Oct 2026

Neural Operator based Reinforcement Learning for Control of first-order PDEs with Spatially-Varying State Delay

arXiv 2501.18201 ↗First posted 30 Jan 2025cs.AI cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Control of distributed parameter systems affected by delays is a challenging task, particularly when the delays depend on spatial variables. The idea of integrating analytical control theory with learning-based control within a unified control scheme is becoming increasingly promising and advantageous. In this paper, we address the problem of controlling an unstable first-order hyperbolic PDE with spatially-varying delays by combining PDE backstepping control strategies and deep reinforcement learning (RL). To eliminate the assumption on the delay function required for the backstepping design, we propose a soft actor-critic (SAC) architecture incorporating a DeepONet to approximate the backstepping controller. The DeepONet extracts features from the backstepping controller and feeds them into the policy network. In simulations, our algorithm outperforms the baseline SAC without prior backstepping knowledge and the analytical controller.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 14 references that are in the map. First posted on arXiv 30 Jan 2025.

Builds on

8 of 14 references.

References · 8

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2023Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learningcited by 111
  3. 2023Neural Operators for Bypassing Gain and Control Computations in PDE Backsteppingcited by 29
  4. 2021Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuningcited by 24
  5. 2023Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learningcited by 24
  6. 2020Reinforcement Learning with Fast Stabilization in Linear Dynamical Systemscited by 20
  7. 2023Neural Operators for PDE Backstepping Control of First-Order Hyperbolic PIDE with Recycle and Delaycited by 13
  8. 2022Proximal Policy Optimization Learning based Control of Congested Freeway Trafficcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs
  2. 2026Backstepping-Guided Reinforcement Learning for Wide-Range Saint-Venant Canal Regulation
  3. 2024Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems
  4. 2026Hyperfastrl: Hypernetwork-based reinforcement learning for unified control of parametric chaotic PDEs
  5. 2024Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies
  6. 2025HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systems
  7. 2026CINOC: Cardinality-Invariant Neural Operator Policies for Scalable PDE Control ↗
  8. 2024Control-Theoretic Techniques for Online Adaptation of Deep Neural Networks in Dynamical Systems