# CityLearn: Standardizing Research in Multi-Agent Reinforcement Learning for Demand Response and Urban Energy Management

arXiv: https://arxiv.org/abs/2012.10504 · HTML: https://mlsanity.com/paper/2012.10504/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2020.

## Built on by (most cited first, 27 of 27)

- [Application-Driven Innovation in Machine Learning](https://mlsanity.com/paper/2403.17381/) (2024)
- [PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systems](https://mlsanity.com/paper/2111.05969/) (2021)
- [GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Management](https://mlsanity.com/paper/2110.06396/) (2021)
- [Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings](https://mlsanity.com/paper/2112.06127/) (2021)
- [RL2Grid: Benchmarking Reinforcement Learning in Power Grid Operations](https://mlsanity.com/paper/2503.23101/) (2025)
- [Off-the-Grid MARL: Datasets with Baselines for Offline Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.00521/) (2023)
- [Winning the CityLearn Challenge: Adaptive Optimization with Evolutionary Search under Trajectory-based Guidance](https://mlsanity.com/paper/2212.01939/) (2022)
- [Federated Ensemble-Directed Offline Reinforcement Learning](https://mlsanity.com/paper/2305.03097/) (2023)
- [Performance Comparison of Deep RL Algorithms for Energy Systems Optimal Scheduling](https://mlsanity.com/paper/2208.00728/) (2022)
- [MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearn](https://mlsanity.com/paper/2301.01148/) (2022)
- [CommonPower: A Framework for Safe Data-Driven Smart Grid Control](https://mlsanity.com/paper/2406.03231/) (2024)
- [RL-ADN: A High-Performance Deep Reinforcement Learning Environment for Optimal Energy Storage Systems Dispatch in Active Distribution Networks](https://mlsanity.com/paper/2408.03685/) (2024)
- [EnergAIze: Multi Agent Deep Deterministic Policy Gradient for Vehicle to Grid Energy Management](https://mlsanity.com/paper/2404.02361/) (2024)
- [Multi-Agent Reinforcement Learning for Fast-Timescale Demand Response of Residential Loads](https://mlsanity.com/paper/2301.02593/) (2023)
- [Impact of data for forecasting on performance of model predictive control in buildings with smart energy storage](https://mlsanity.com/paper/2402.12539/) (2024)
- [B2RL: An open-source Dataset for Building Batch Reinforcement Learning](https://mlsanity.com/paper/2209.15626/) (2022)
- [BEAR: Physics-Principled Building Environment for Control and Reinforcement Learning](https://mlsanity.com/paper/2211.14744/) (2022)
- [Scalable Constrained Multi-Agent Reinforcement Learning via State Augmentation and Consensus for Separable Dynamics](https://mlsanity.com/paper/2605.30461/) (2026)
- [Adaptive Control of Resource Flow to Optimize Construction Work and Cash Flow via Online Deep Reinforcement Learning](https://mlsanity.com/paper/2307.10574/) (2023)
- [A Novel Bifurcation Method for Observation Perturbation Attacks on Reinforcement Learning Agents: Load Altering Attacks on a Cyber Physical Power System](https://mlsanity.com/paper/2407.05182/) (2024)
- [Optimization Solution Functions as Deterministic Policies for Offline Reinforcement Learning](https://mlsanity.com/paper/2408.15368/) (2024)
- [BuildingGym: An open-source toolbox for AI-based building energy management using reinforcement learning](https://mlsanity.com/paper/2509.11922/) (2025)
- [MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets](https://mlsanity.com/paper/2602.16063/) (2026)
- [Characterizing MARL for Energy Control: A Multi-KPI Benchmark on the CityLearn Environment](https://mlsanity.com/paper/2602.19223/) (2026)
- [From Pixels to Digital Agents: An Empirical Study on the Taxonomy and Technological Trends of Reinforcement Learning Environments](https://mlsanity.com/paper/2603.23964/) (2026)
- [PIRS: Physics-Informed Reward Shaping for SAC-Based Building Energy Management](https://mlsanity.com/paper/2605.28232/) (2026)
- [CityLearn v3: A Configurable Simulation and Evaluation Framework for Realistic Control Studies of Renewable Energy Communities](https://mlsanity.com/paper/2609.21570/) (2026)

## Abstract (arXiv)

Rapid urbanization, increasing integration of distributed renewable energy resources, energy storage, and electric vehicles introduce new challenges for the power grid. In the US, buildings represent about 70% of the total electricity demand and demand response has the potential for reducing peaks of electricity by about 20%. Unlocking this potential requires control systems that operate on distributed systems, ideally data-driven and model-free. For this, reinforcement learning (RL) algorithms have gained increased interest in the past years. However, research in RL for demand response has been lacking the level of standardization that propelled the enormous progress in RL research in the computer science community. To remedy this, we created CityLearn, an OpenAI Gym Environment which allows researchers to implement, share, replicate, and compare their implementations of RL for demand response. Here, we discuss this environment and The CityLearn Challenge, a RL competition we organized to propel further progress in this field.