mlsanityData to 14 Oct 2026

Independent policy gradient-based reinforcement learning for economic and reliable energy management of multi-microgrid systems

arXiv 2511.20977 ↗First posted 26 Nov 2025eess.SY cs.LG cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Efficiency and reliability are both crucial for energy management, especially in multi-microgrid systems (MMSs) integrating intermittent and distributed renewable energy sources. This study investigates an economic and reliable energy management problem in MMSs under a distributed scheme, where each microgrid independently updates its energy management policy in a decentralized manner to optimize the long-term system performance collaboratively. We introduce the mean and variance of the exchange power between the MMS and the main grid as indicators for the economic performance and reliability of the system. Accordingly, we formulate the energy management problem as a mean-variance team stochastic game (MV-TSG), where conventional methods based on the maximization of expected cumulative rewards are unsuitable for variance metrics. To solve MV-TSGs, we propose a fully distributed independent policy gradient algorithm, with rigorous convergence analysis, for scenarios with known model parameters. For large-scale scenarios with unknown model parameters, we further develop a deep reinforcement learning algorithm based on independent policy gradients, enabling data-driven policy optimization. Numerical experiments in two scenarios validate the effectiveness of the proposed methods. Our approaches fully leverage the distributed computational capabilities of MMSs and achieve a well-balanced trade-off between economic performance and operational reliability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 51 references that are in the map. First posted on arXiv 26 Nov 2025.

Builds on

21 of 51 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  7. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  8. 2014Convex Optimization: Algorithms and Complexitycited by 575
  9. 2021Global Convergence of Multi-Agent Policy Gradient in Markov Potential Gamescited by 93
  10. 2023Heterogeneous-Agent Reinforcement Learningcited by 69
  11. 2019Risk-Averse Trust Region Optimization for Reward-Volatility Reductioncited by 54
  12. 2021Gradient play in stochastic games: stationary points, convergence, and sample complexitycited by 34
9 more
  1. 2021RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agentscited by 31
  2. 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 20
  3. 2021Average-Reward Reinforcement Learning with Trust Region Methodscited by 19
  4. 2020Risk-Sensitive Markov Decision Processes with Combined Metrics of Mean and Variancecited by 9
  5. 2016Stochastic Games for Smart Grid Energy Management with Prospect Prosumerscited by 6
  6. 2024Provable Policy Gradient Methods for Average-Reward Markov Potential Gamescited by 5
  7. 2022A unified algorithm framework for mean-variance optimization in discounted Markov decision processescited by 3
  8. 2022Combined Peak Reduction and Self-Consumption Using Proximal Policy Optimizationcited by 1
  9. 2024Scalable Adaptive Traffic Light Control Over a Traffic Network Including Turns, Transit Delays, and Blockingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Reinforcement Learning Approach for Optimal Control in Microgrids
  2. 2024A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning
  3. 2025AutoGrid AI: Deep Reinforcement Learning Framework for Autonomous Microgrid Management
  4. 2026Supervised Reinforcement Learning for the Coordination of Distributed Energy Resources
  5. 2025A New Error Temporal Difference Algorithm for Deep Reinforcement Learning in Microgrid Optimization
  6. 2026Multiagent Reinforcement Learning in Enhancing Resilience of Microgrids under Extreme Weather Events ↗
  7. 2025Cooperative Energy Scheduling of Multi-Microgrids Based on Risk-Sensitive Reinforcement Learning
  8. 2025Diffusion-Modeled Reinforcement Learning for Carbon and Risk-Aware Microgrid Optimization