# Optimizing Energy Management of Smart Grid using Reinforcement Learning aided by Surrogate models built using Physics-informed Neural Networks

arXiv: https://arxiv.org/abs/2510.17380 · HTML: https://mlsanity.com/paper/2510.17380/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 51 references that are in the map.
- First posted on arXiv 20 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 11 of 11)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Physics-Informed Neural Networks for AC Optimal Power Flow](https://mlsanity.com/paper/2110.02672/) (2021)
- [Leveraging power grid topology in machine learning assisted optimal power flow](https://mlsanity.com/paper/2110.00306/) (2021)
- [Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systems](https://mlsanity.com/paper/2103.07932/) (2021)
- [Unsupervised Optimal Power Flow Using Graph Neural Networks](https://mlsanity.com/paper/2210.09277/) (2022)
- [PINN surrogate of Li-ion battery models for parameter inference. Part I: Implementation and multi-fidelity hierarchies for the single-particle model](https://mlsanity.com/paper/2312.17329/) (2023)
- [Constrained Reinforcement Learning for Predictive Control in Real-Time Stochastic Dynamic Optimal Power Flow](https://mlsanity.com/paper/2302.10382/) (2023)
- [PINN surrogate of Li-ion battery models for parameter inference. Part II: Regularization and application of the pseudo-2D model](https://mlsanity.com/paper/2312.17336/) (2023)
- [A Reinforcement Learning Approach to Dairy Farm Battery Management using Q Learning](https://mlsanity.com/paper/2403.09499/) (2024)
- [KKT-Informed Neural Network](https://mlsanity.com/paper/2409.09087/) (2024)
- [Building surrogate models using trajectories of agents trained by Reinforcement Learning](https://mlsanity.com/paper/2509.01285/) (2025)

## Abstract (arXiv)

Optimizing the energy management within a smart grids scenario presents significant challenges, primarily due to the complexity of real-world systems and the intricate interactions among various components. Reinforcement Learning (RL) is gaining prominence as a solution for addressing the challenges of Optimal Power Flow in smart grids. However, RL needs to iterate compulsively throughout a given environment to obtain the optimal policy. This means obtaining samples from a, most likely, costly simulator, which can lead to a sample efficiency problem. In this work, we address this problem by substituting costly smart grid simulators with surrogate models built using Phisics-informed Neural Networks (PINNs), optimizing the RL policy training process by arriving to convergent results in a fraction of the time employed by the original environment.