mlsanityData to 14 Oct 2026

Characterizing MARL for Energy Control: A Multi-KPI Benchmark on the CityLearn Environment

arXiv 2602.19223 ↗First posted 22 Feb 2026cs.AI cs.LG cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

The optimization of urban energy systems is crucial for the advancement of sustainable and resilient smart cities, which are becoming increasingly complex with multiple decision-making units. To address scalability and coordination concerns, Multi-Agent Reinforcement Learning (MARL) is a promising solution. This paper addresses the imperative need for comprehensive and reliable benchmarking of MARL algorithms on energy management tasks. CityLearn is used as a case study environment because it realistically simulates urban energy systems, incorporates multiple storage systems, and utilizes renewable energy sources. By doing so, our work sets a new standard for evaluation, conducting a comparative study across multiple key performance indicators (KPIs). This approach illuminates the key strengths and weaknesses of various algorithms, moving beyond traditional KPI averaging which often masks critical insights. Our experiments utilize widely accepted baselines such as Proximal Policy Optimization (PPO) and Soft Actor Critic (SAC), and encompass diverse training schemes including Decentralized Training with Decentralized Execution (DTDE) and Centralized Training with Decentralized Execution (CTDE) approaches and different neural network architectures. Our work also proposes novel KPIs that tackle real world implementation challenges such as individual building contribution and battery storage lifetime. Our findings show that DTDE consistently outperforms CTDE in both average and worst-case performance. Additionally, temporal dependency learning improved control on memory dependent KPIs such as ramping and battery usage, contributing to more sustainable battery operation. Results also reveal robustness to agent or resource removal, highlighting both the resilience and decentralizability of the learned policies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 34 references that are in the map. First posted on arXiv 22 Feb 2026.

Builds on

17 of 34 references.

References · 17

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  4. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  5. 2022Multi-Agent Reinforcement Learning is a Sequence Modeling Problemcited by 155
  6. 2015Memory-based control with recurrent neural networkscited by 78
  7. 2019Measuring the Reliability of Reinforcement Learning Algorithmscited by 51
  8. 2022Towards a Standardised Performance Evaluation Protocol for Cooperative MARLcited by 49
  9. 2018GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithmscited by 40
  10. 2021Podracer architectures for scalable Reinforcement Learningcited by 38
  11. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 32
  12. 2020CityLearn: Standardizing Research in Multi-Agent Reinforcement Learning for Demand Response and Urban Energy Managementcited by 27
5 more
  1. 2025Multi-Agent Reinforcement Learning for Resources Allocation Optimization: A Surveycited by 14
  2. 2024Sable: a Performant, Efficient and Scalable Sequence Model for MARLcited by 10
  3. 2020A Centralised Soft Actor Critic Deep Reinforcement Learning Approach to District Demand Side Management through CityLearncited by 7
  4. 2022Winning the CityLearn Challenge: Adaptive Optimization with Evolutionary Search under Trajectory-based Guidancecited by 7
  5. 2023Efficiently Quantifying Individual Agent Importance in Cooperative MARLcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Multi-Agent Reinforcement Learning for Energy Networks: Computational Challenges, Progress and Open Problems
  2. 2024CityLearn v2: Energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communities
  3. 2024Reinforcement Learning for Sustainable Energy: A Survey
  4. 2025Agent-Based Decentralized Energy Management of EV Charging Station with Solar Photovoltaics via Multi-Agent Reinforcement Learning
  5. 2026Knowledge Distillation for Efficient Transformer-Based Reinforcement Learning in Hardware-Constrained Energy Management Systems
  6. 2025Multi-Agent Reinforcement Learning in Intelligent Transportation Systems: A Comprehensive Survey
  7. 2026Supervised Reinforcement Learning for the Coordination of Distributed Energy Resources
  8. 2026MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets