MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets
Not on the map. Why?
Abstract (arXiv)
This paper introduces a novel, open-source MARL simulation framework for studying implicit cooperation in LEMs, modeled as a decentralized partially observable Markov decision process and implemented as a Gymnasium environment for MARL. Our framework features a modular market platform with plug-and-play clearing mechanisms, physically constrained agent models (including battery storage), a realistic grid network, and a comprehensive analytics suite to evaluate emergent coordination. The main contribution is a novel method to foster implicit cooperation, where agents' observations and rewards are enhanced with system-level key performance indicators to enable them to independently learn strategies that benefit the entire system and aim for collectively beneficial outcomes without explicit communication. Through representative case studies (available in a dedicated GitHub repository in https://github.com/salazarna/marlem, we show the framework's ability to analyze how different market configurations (such as varying storage deployment) impact system performance. This illustrates its potential to facilitate emergent coordination, improve market efficiency, and strengthen grid stability. The proposed simulation framework is a flexible, extensible, and reproducible tool for researchers and practitioners to design, test, and validate strategies for future intelligent, decentralized energy systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 56 references that are in the map. First posted on arXiv 17 Feb 2026.
Builds on
20 of 56 references.
References · 20
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2019The StarCraft Multi-Agent Challengecited by 589
- 2017Population Based Training of Neural Networkscited by 351
- 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 207
- 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
- 2017pandapower - an Open Source Python Tool for Convenient Modeling, Analysis and Optimization of Electric Power Systemscited by 79
- 2020Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learningcited by 58
- 2021Learning to run a Power Network Challenge: a Retrospective Analysiscited by 52
- 2020CityLearn: Standardizing Research in Multi-Agent Reinforcement Learning for Demand Response and Urban Energy Managementcited by 27
- 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
- 2021Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systemscited by 18
- 2021PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systemscited by 15
8 more
- 2024CityLearn v2: Energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communitiescited by 14
- 2018Learning to Communicate Implicitly By Actionscited by 11
- 2023Centralised rehearsal of decentralised cooperation: Multi-agent reinforcement learning for the scalable coordination of residential energy flexibilitycited by 5
- 2024EVLearn: Extending the CityLearn Framework with Electric Vehicle Simulationcited by 5
- 2018Decentralized P2P Energy Trading under Network Constraints in a Low-Voltage Networkcited by 4
- 2023MARL-iDR: Multi-Agent Reinforcement Learning for Incentive-based Residential Demand Responsecited by 4
- 2024RL-ADN: A High-Performance Deep Reinforcement Learning Environment for Optimal Energy Storage Systems Dispatch in Active Distribution Networkscited by 3
- 2024Multi-Agent Reinforcement Learning for Energy Networks: Computational Challenges, Progress and Open Problemscited by 2
Built on by
Not cited yet.