# Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs

arXiv: https://arxiv.org/abs/2507.04232 · HTML: https://mlsanity.com/paper/2507.04232/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 6 Jul 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 14 of 14)

- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [LyaNet: A Lyapunov Framework for Training Neural ODEs](https://mlsanity.com/paper/2202.02526/) (2022)
- [Neural Operators for Bypassing Gain and Control Computations in PDE Backstepping](https://mlsanity.com/paper/2302.14265/) (2023)
- [Neural Operators of Backstepping Controller and Observer Gain Functions for Reaction-Diffusion PDEs](https://mlsanity.com/paper/2303.10506/) (2023)
- [Neural Operators for PDE Backstepping Control of First-Order Hyperbolic PIDE with Recycle and Delay](https://mlsanity.com/paper/2307.11436/) (2023)
- [PDE Control Gym: A Benchmark for Data-Driven Boundary Control of Partial Differential Equations](https://mlsanity.com/paper/2405.11401/) (2024)
- [Control-Informed Reinforcement Learning for Chemical Processes](https://mlsanity.com/paper/2408.13566/) (2024)
- [Deep Learning of Delay-Compensated Backstepping for Reaction-Diffusion PDEs](https://mlsanity.com/paper/2308.10501/) (2023)
- [Adaptive Neural-Operator Backstepping Control of a Benchmark Hyperbolic PDE](https://mlsanity.com/paper/2401.07862/) (2024)
- [Online identification and control of PDEs via Reinforcement Learning methods](https://mlsanity.com/paper/2308.04068/) (2023)
- [Adaptive control of reaction-diffusion PDEs via neural operator-approximated gain kernels](https://mlsanity.com/paper/2407.01745/) (2024)
- [Stochastic Reinforcement Learning with Stability Guarantees for Control of Unknown Nonlinear Systems](https://mlsanity.com/paper/2409.08382/) (2024)
- [Neural Operator Feedback for a First-Order PIDE with Spatially-Varying State Delay](https://mlsanity.com/paper/2412.08219/) (2024)

## Abstract (arXiv)

This paper develops a reinforcement learning-based controller for the stabilization of partial differential equation (PDE) systems. Within the soft actor-critic (SAC) framework, we embed a DeepONet, a well-known neural operator (NO), which is pretrained using the backstepping controller. The pretrained DeepONet captures the essential features of the backstepping controller and serves as a feature extractor, replacing the convolutional neural networks (CNNs) layers in the original actor and critic networks, and directly connects to the fully connected layers of the SAC architecture. We apply this novel backstepping and reinforcement learning integrated method to stabilize an unstable ffrst-order hyperbolic PDE and an unstable reactiondiffusion PDE. Simulation results demonstrate that the proposed method outperforms the standard SAC, SAC with an untrained DeepONet, and the backstepping controller on both systems.