mlsanityData to 14 Oct 2026

CityLearn v2: Energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communities

arXiv 2405.03848 ↗First posted 2 May 2024cs.LG cs.CY cs.SYcited by 15

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

As more distributed energy resources become part of the demand-side infrastructure, it is important to quantify the energy flexibility they provide on a community scale, particularly to understand the impact of geographic, climatic, and occupant behavioral differences on their effectiveness, as well as identify the best control strategies to accelerate their real-world adoption. CityLearn provides an environment for benchmarking simple and advanced distributed energy resource control algorithms including rule-based, model-predictive, and reinforcement learning control. CityLearn v2 presented here extends CityLearn v1 by providing a simulation environment that leverages the End-Use Load Profiles for the U.S. Building Stock dataset to create virtual grid-interactive communities for resilient, multi-agent distributed energy resources and objective control with dynamic occupant feedback. This work details the v2 environment design and provides application examples that utilize reinforcement learning to manage battery energy storage system charging/discharging cycles, vehicle-to-grid control, and thermal comfort during heat pump power modulation.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 117 references that are in the map. First posted on arXiv 2 May 2024.

Builds on

15 of 117 references.

References · 15

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2016OpenAI Gymcited by 1,736
  3. 2019Tackling Climate Change with Machine Learningcited by 265
  4. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learningcited by 63
  5. 2019A Review of Reinforcement Learning for Autonomous Building Energy Managementcited by 21
  6. 2020The ASHRAE Great Energy Predictor III competition: Overview and resultscited by 15
  7. 2021GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Managementcited by 12
  8. 2021Real-world challenges for multi-agent reinforcement learning in grid-interactive buildingscited by 9
  9. 2020A Centralised Soft Actor Critic Deep Reinforcement Learning Approach to District Demand Side Management through CityLearncited by 7
  10. 2024EVLearn: Extending the CityLearn Framework with Electric Vehicle Simulationcited by 5
  11. 2022MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearncited by 4
  12. 2022Data-Driven Key Performance Indicators and Datasets for Building Energy Flexibility: A Review and Perspectivescited by 3
3 more
  1. 2024EnergAIze: Multi Agent Deep Deterministic Policy Gradient for Vehicle to Grid Energy Managementcited by 3
  2. 2024Impact of data for forecasting on performance of model predictive control in buildings with smart energy storagecited by 2
  3. 2021MARTINI: Smart Meter Driven Estimation of HVAC Schedules and Energy Savings Based on WiFi Sensing and Clusteringcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 4; 2026: 8.

15 papers.

Sort:

By area:

  1. 2025Adapting to Change: A Comparison of Continual and Transfer Learning for Modeling Building Thermal Dynamics under Concept Driftscited by 7
  2. 2024SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learningcited by 5
  3. 2024CommonPower: A Framework for Safe Data-Driven Smart Grid Controlcited by 4
  4. 2025Control of Renewable Energy Communities using AI and Real-World Datacited by 1
  5. 2026Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learningcited by 1
  6. 2024Generalising Battery Control in Net-Zero Buildings via Personalised Federated RLcited by 0
  7. 2025Physics-informed Modularized Neural Network for Advanced Building Control by Deep Reinforcement Learningcited by 0
  8. 2025STEMS: Spatial-Temporal Enhanced Safe Multi-Agent Coordination for Building Energy Managementcited by 0
  9. 2026Representation Learning Enhanced Deep Reinforcement Learning for Optimal Operation of Hydrogen-based Multi-Energy Systemscited by 0
  10. 2026MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Marketscited by 0
  11. 2026End-to-End Learning-based Operation of Integrated Energy Systems for Buildings and Data Centerscited by 0
  12. 2026OccuReward: LLM-Guided Occupant-Centric Reward Shaping for Demographic Equity in Grid-Interactive Buildingscited by 0
3 more
  1. 2026Contextual Quality-Diversity Evolutionary Reinforcement Learning for HVAC Control in Tropical Commercial Buildingscited by 0
  2. 2026Large Language Models for HVAC Operations in Building Energy Systems: A Critical Review of Methods, Applications, and Deployment Readinesscited by 0
  3. 2026CityLearn v3: A Configurable Simulation and Evaluation Framework for Realistic Control Studies of Renewable Energy Communitiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Applications in CityLearn Gym Environment for Multi-Objective Control Benchmarking in Grid-Interactive Buildings and Districts ↗
  2. 2024Reinforcement Learning for Sustainable Energy: A Survey
  3. 2026Characterizing MARL for Energy Control: A Multi-KPI Benchmark on the CityLearn Environment
  4. 2026Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings
  5. 2026Knowledge Distillation for Efficient Transformer-Based Reinforcement Learning in Hardware-Constrained Energy Management Systems
  6. 2026AutoB2G: Agentic Simulation and Reinforcement Learning for Spatio-Temporal Grid-Interactive Building Control
  7. 2024Carbon Footprint Reduction for Sustainable Data Centers in Real-Time
  8. 2026Verifier-Based Reinforcement Fine-Tuning of Reasoning Models for Thermal Energy Storage Control