# LLM-Enhanced Multi-Agent Reinforcement Learning with Expert Workflow for Real-Time P2P Energy Trading

arXiv: https://arxiv.org/abs/2507.14995 · HTML: https://mlsanity.com/paper/2507.14995/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 68 references that are in the map.
- First posted on arXiv 20 Jul 2025.

## Placed in

- LLM agents and tool use / [Financial trading and forecasting agents](https://mlsanity.com/agents/financial-trading-and-forecasting-agents/)

## Builds on (most cited first, 24 of 24)

- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling](https://mlsanity.com/paper/2405.17743/) (2024)
- [Multi-Agent Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1807.09936/) (2018)
- [A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity](https://mlsanity.com/paper/1707.09183/) (2017)
- [Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving](https://mlsanity.com/paper/2103.10690/) (2021)
- [Making Efficient Use of Demonstrations to Solve Hard Exploration Problems](https://mlsanity.com/paper/1909.01387/) (2019)
- [BenchMARL: Benchmarking Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2312.01472/) (2023)
- [Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?](https://mlsanity.com/paper/2305.17352/) (2023)
- [Large Foundation Models for Power Systems](https://mlsanity.com/paper/2312.07044/) (2023)
- [Enhancing LLMs for Power System Simulations: A Feedback-driven Multi-agent Framework](https://mlsanity.com/paper/2411.16707/) (2024)
- [TeLL-Drive: Enhancing Autonomous Driving with Teacher LLM-Guided Deep Reinforcement Learning](https://mlsanity.com/paper/2502.01387/) (2025)
- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [A Deeper Understanding of State-Based Critics in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2201.01221/) (2022)
- [RL2: Reinforce Large Language Model to Assist Safe Reinforcement Learning for Energy Management of Active Distribution Networks](https://mlsanity.com/paper/2412.01303/) (2024)
- [Large Language Model guided Deep Reinforcement Learning for Decision Making in Autonomous Driving](https://mlsanity.com/paper/2412.18511/) (2024)
- [Leveraging LLM-Assisted Query Understanding for Live Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.21384/) (2025)
- [Distributed Energy Trading and Scheduling among Microgrids via Multiagent Reinforcement Learning](https://mlsanity.com/paper/2007.04517/) (2020)
- [Hybrid Imitation Learning for Real-Time Service Restoration in Resilient Distribution Systems](https://mlsanity.com/paper/2011.14458/) (2020)
- [Peer-to-Peer Energy Trading of Solar and Energy Storage: A Networked Multiagent Reinforcement Learning Approach](https://mlsanity.com/paper/2401.13947/) (2024)
- [Online distributed optimization for spatio-temporally constrained real-time peer-to-peer energy trading](https://mlsanity.com/paper/2505.21884/) (2025)

## Built on by (most cited first, 2 of 2)

- [A MARL Centered Reference Architecture for Large Language Model Augmentation in Smart Manufacturing](https://mlsanity.com/paper/2608.07148/) (2026)
- [Grounded, Compute-Efficient LLM Policy Agents for Energy-Poverty Equity in Physically-Constrained Peer-to-Peer Energy Markets](https://mlsanity.com/paper/2609.01918/) (2026)

## Abstract (arXiv)

Real-time peer-to-peer (P2P) electricity markets dynamically adapt to fluctuations in renewable energy and variations in demand, maximizing economic benefits through instantaneous price responses while enhancing grid flexibility. However, scaling expert guidance for massive personalized prosumers poses critical challenges, including diverse decision-making demands and a lack of customized modeling frameworks. This paper proposes an integrated large language model-multi-agent reinforcement learning (LLM-MARL) framework for real-time P2P energy trading to address challenges such as the limited technical capability of prosumers, the lack of expert experience, and security issues of distribution networks. LLMs are introduced as experts to generate personalized strategies, guiding MARL under the centralized training with decentralized execution (CTDE) paradigm through imitation. To handle the scalability issues inherent in large-scale P2P networks, a differential attention-based critic network is introduced to efficiently extract key interaction features and enhance convergence. Experimental results demonstrate that LLM-generated strategies effectively substitute human experts. The proposed imitative expert MARL algorithms achieve significantly lower economic costs and voltage violation rates on test sets compared to baseline algorithms, while maintaining robust stability. This paper provides an effective solution for the real-time decision-making of the P2P electricity market by bridging expert knowledge with agent learning.