# CityLearn v2: Energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communities

arXiv: https://arxiv.org/abs/2405.03848 · HTML: https://mlsanity.com/paper/2405.03848/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 117 references that are in the map.
- First posted on arXiv 2 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 15 of 15)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Tackling Climate Change with Machine Learning](https://mlsanity.com/paper/1906.05433/) (2019)
- [NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning](https://mlsanity.com/paper/2102.00714/) (2021)
- [A Review of Reinforcement Learning for Autonomous Building Energy Management](https://mlsanity.com/paper/1903.05196/) (2019)
- [The ASHRAE Great Energy Predictor III competition: Overview and results](https://mlsanity.com/paper/2007.06933/) (2020)
- [GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Management](https://mlsanity.com/paper/2110.06396/) (2021)
- [Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings](https://mlsanity.com/paper/2112.06127/) (2021)
- [A Centralised Soft Actor Critic Deep Reinforcement Learning Approach to District Demand Side Management through CityLearn](https://mlsanity.com/paper/2009.10562/) (2020)
- [EVLearn: Extending the CityLearn Framework with Electric Vehicle Simulation](https://mlsanity.com/paper/2404.06521/) (2024)
- [MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearn](https://mlsanity.com/paper/2301.01148/) (2022)
- [Data-Driven Key Performance Indicators and Datasets for Building Energy Flexibility: A Review and Perspectives](https://mlsanity.com/paper/2211.12252/) (2022)
- [EnergAIze: Multi Agent Deep Deterministic Policy Gradient for Vehicle to Grid Energy Management](https://mlsanity.com/paper/2404.02361/) (2024)
- [Impact of data for forecasting on performance of model predictive control in buildings with smart energy storage](https://mlsanity.com/paper/2402.12539/) (2024)
- [MARTINI: Smart Meter Driven Estimation of HVAC Schedules and Energy Savings Based on WiFi Sensing and Clustering](https://mlsanity.com/paper/2110.08927/) (2021)

## Built on by (most cited first, 15 of 15)

- [Adapting to Change: A Comparison of Continual and Transfer Learning for Modeling Building Thermal Dynamics under Concept Drifts](https://mlsanity.com/paper/2508.21615/) (2025)
- [SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learning](https://mlsanity.com/paper/2412.08293/) (2024)
- [CommonPower: A Framework for Safe Data-Driven Smart Grid Control](https://mlsanity.com/paper/2406.03231/) (2024)
- [Control of Renewable Energy Communities using AI and Real-World Data](https://mlsanity.com/paper/2505.17321/) (2025)
- [Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning](https://mlsanity.com/paper/2607.16534/) (2026)
- [Generalising Battery Control in Net-Zero Buildings via Personalised Federated RL](https://mlsanity.com/paper/2412.20946/) (2024)
- [Physics-informed Modularized Neural Network for Advanced Building Control by Deep Reinforcement Learning](https://mlsanity.com/paper/2504.05397/) (2025)
- [STEMS: Spatial-Temporal Enhanced Safe Multi-Agent Coordination for Building Energy Management](https://mlsanity.com/paper/2510.14112/) (2025)
- [Representation Learning Enhanced Deep Reinforcement Learning for Optimal Operation of Hydrogen-based Multi-Energy Systems](https://mlsanity.com/paper/2602.00027/) (2026)
- [MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets](https://mlsanity.com/paper/2602.16063/) (2026)
- [End-to-End Learning-based Operation of Integrated Energy Systems for Buildings and Data Centers](https://mlsanity.com/paper/2604.14184/) (2026)
- [OccuReward: LLM-Guided Occupant-Centric Reward Shaping for Demographic Equity in Grid-Interactive Buildings](https://mlsanity.com/paper/2605.28168/) (2026)
- [Contextual Quality-Diversity Evolutionary Reinforcement Learning for HVAC Control in Tropical Commercial Buildings](https://mlsanity.com/paper/2608.11324/) (2026)
- [Large Language Models for HVAC Operations in Building Energy Systems: A Critical Review of Methods, Applications, and Deployment Readiness](https://mlsanity.com/paper/2609.05314/) (2026)
- [CityLearn v3: A Configurable Simulation and Evaluation Framework for Realistic Control Studies of Renewable Energy Communities](https://mlsanity.com/paper/2609.21570/) (2026)

## Abstract (arXiv)

As more distributed energy resources become part of the demand-side infrastructure, it is important to quantify the energy flexibility they provide on a community scale, particularly to understand the impact of geographic, climatic, and occupant behavioral differences on their effectiveness, as well as identify the best control strategies to accelerate their real-world adoption. CityLearn provides an environment for benchmarking simple and advanced distributed energy resource control algorithms including rule-based, model-predictive, and reinforcement learning control. CityLearn v2 presented here extends CityLearn v1 by providing a simulation environment that leverages the End-Use Load Profiles for the U.S. Building Stock dataset to create virtual grid-interactive communities for resilient, multi-agent distributed energy resources and objective control with dynamic occupant feedback. This work details the v2 environment design and provides application examples that utilize reinforcement learning to manage battery energy storage system charging/discharging cycles, vehicle-to-grid control, and thermal comfort during heat pump power modulation.