# RL2: Reinforce Large Language Model to Assist Safe Reinforcement Learning for Energy Management of Active Distribution Networks

arXiv: https://arxiv.org/abs/2412.01303 · HTML: https://mlsanity.com/paper/2412.01303/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 40 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Risk-Constrained Reinforcement Learning with Percentile Risk Criteria](https://mlsanity.com/paper/1512.01629/) (2015)
- [Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks](https://mlsanity.com/paper/2110.14300/) (2021)
- [Exploring the Capabilities and Limitations of Large Language Models in the Electric Energy Sector](https://mlsanity.com/paper/2403.09125/) (2024)
- [Large Foundation Models for Power Systems](https://mlsanity.com/paper/2312.07044/) (2023)
- [Online Multi-agent Reinforcement Learning for Decentralized Inverter-based Volt-VAR Control](https://mlsanity.com/paper/2006.12841/) (2020)
- [Enabling Large Language Models to Perform Power System Simulations with Previously Unseen Tools: A Case of Daline](https://mlsanity.com/paper/2406.17215/) (2024)

## Built on by (most cited first, 9 of 9)

- [Large Language Model Powered Automated Modeling and Optimization of Active Distribution Network Dispatch Problems](https://mlsanity.com/paper/2507.21162/) (2025)
- [Large Language Model-Empowered Interactive Load Forecasting](https://mlsanity.com/paper/2505.16577/) (2025)
- [Large Language Model-enhanced Reinforcement Learning for Low-Altitude Economy Networking](https://mlsanity.com/paper/2505.21045/) (2025)
- [LLM-Enhanced Multi-Agent Reinforcement Learning with Expert Workflow for Real-Time P2P Energy Trading](https://mlsanity.com/paper/2507.14995/) (2025)
- [Two-Stage Active Distribution Network Voltage Control via LLM-RL Collaboration: A Hybrid Knowledge-Data-Driven Approach](https://mlsanity.com/paper/2602.21715/) (2026)
- [Foundation-Model-Based Agents in Industrial Automation: Purposes, Capabilities, and Open Challenges](https://mlsanity.com/paper/2605.02592/) (2026)
- [Large Language Model as An Operator: An Experience-Driven Solution for Distribution Network Voltage Control](https://mlsanity.com/paper/2507.14800/) (2025)
- [One Request, Multiple Experts: LLM Orchestrates Domain Specific Models via Adaptive Task Routing](https://mlsanity.com/paper/2511.12484/) (2025)
- [LLMs are Few-Shot Decision-Makers: Generalized Context-Aware Microgrid Frequency Control through Prompt Decision Transformer](https://mlsanity.com/paper/2608.21858/) (2026)

## Abstract (arXiv)

As large-scale distributed energy resources are integrated into the active distribution networks (ADNs), effective energy management in ADNs becomes increasingly prominent compared to traditional distribution networks. Although advanced reinforcement learning (RL) methods, which alleviate the burden of complicated modelling and optimization, have greatly improved the efficiency of energy management in ADNs, safety becomes a critical concern for RL applications in real-world problems. Since the design and adjustment of penalty functions, which correspond to operational safety constraints, requires extensive domain knowledge in RL and power system operation, the emerging ADN operators call for a more flexible and customized approach to address the penalty functions so that the operational safety and efficiency can be further enhanced. Empowered with strong comprehension, reasoning, and in-context learning capabilities, large language models (LLMs) provide a promising way to assist safe RL for energy management in ADNs. In this paper, we introduce the LLM to comprehend operational safety requirements in ADNs and generate corresponding penalty functions. In addition, we propose an RL2 mechanism to refine the generated functions iteratively and adaptively through multi-round dialogues, in which the LLM agent adjusts the functions' pattern and parameters based on training and test performance of the downstream RL agent. The proposed method significantly reduces the intervention of the ADN operators. Comprehensive test results demonstrate the effectiveness of the proposed method.