# Actor-Critic Methods using Physics-Informed Neural Networks: Control of a 1D PDE Model for Fluid-Cooled Battery Packs

arXiv: https://arxiv.org/abs/2305.10952 · HTML: https://mlsanity.com/paper/2305.10952/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [Neural Networks with Physics-Informed Architectures and Constraints for Dynamical Systems Modeling](https://mlsanity.com/paper/2109.06407/) (2021)
- [Deep Transformer Q-Networks for Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2206.01078/) (2022)
- [Hamilton-Jacobi Deep Q-Learning for Deterministic Continuous-Time Systems with Lipschitz Continuous Controls](https://mlsanity.com/paper/2010.14087/) (2020)
- [Physics-Informed Neural Nets for Control of Dynamical Systems](https://mlsanity.com/paper/2104.02556/) (2021)
- [Physics-informed Neural Networks-based Model Predictive Control for Multi-link Manipulators](https://mlsanity.com/paper/2109.10793/) (2021)
- [Distributional Hamilton-Jacobi-Bellman Equations for Continuous-Time Reinforcement Learning](https://mlsanity.com/paper/2205.12184/) (2022)
- [Polynomial approximation of high-dimensional Hamilton-Jacobi-Bellman equations and applications to feedback control of semilinear parabolic PDEs](https://mlsanity.com/paper/1702.04400/) (2017)
- [Boundary Control of Coupled Reaction-Advection-Diffusion Systems with Spatially-Varying Coefficients](https://mlsanity.com/paper/1603.04914/) (2016)

## Built on by (most cited first, 2 of 2)

- [PDE-Controller: LLMs for Autoformalization and Reasoning of PDEs](https://mlsanity.com/paper/2502.00963/) (2025)
- [Ensemble based Closed-Loop Optimal Control using Physics-Informed Neural Networks](https://mlsanity.com/paper/2510.18195/) (2025)

## Abstract (arXiv)

This paper proposes an actor-critic algorithm for controlling the temperature of a battery pack using a cooling fluid. This is modeled by a coupled 1D partial differential equation (PDE) with a controlled advection term that determines the speed of the cooling fluid. The Hamilton-Jacobi-Bellman (HJB) equation is a PDE that evaluates the optimality of the value function and determines an optimal controller. We propose an algorithm that treats the value network as a Physics-Informed Neural Network (PINN) to solve for the continuous-time HJB equation rather than a discrete-time Bellman optimality equation, and we derive an optimal controller for the environment that we exploit to achieve optimal control. Our experiments show that a hybrid-policy method that updates the value network using the HJB equation and updates the policy network identically to PPO achieves the best results in the control of this PDE system.