mlsanityData to 14 Oct 2026

MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearn

arXiv 2301.01148 ↗First posted 31 Dec 2022cs.LG cs.AI cs.SYcited by 4

Not on the map. Why?

Abstract (arXiv)

The decarbonization of buildings presents new challenges for the reliability of the electrical grid as a result of the intermittency of renewable energy sources and increase in grid load brought about by end-use electrification. To restore reliability, grid-interactive efficient buildings can provide flexibility services to the grid through demand response. Residential demand response programs are hindered by the need for manual intervention by customers. To maximize the energy flexibility potential of residential buildings, an advanced control architecture is needed. Reinforcement learning is well-suited for the control of flexible resources as it is able to adapt to unique building characteristics compared to expert systems. Yet, factors hindering the adoption of RL in real-world applications include its large data requirements for training, control security and generalizability. Here we address these challenges by proposing the MERLIN framework and using a digital twin of a real-world 17-building grid-interactive residential community in CityLearn. We show that 1) independent RL-controllers for batteries improve building and district level KPIs compared to a reference RBC by tailoring their policies to individual buildings, 2) despite unique occupant behaviours, transferring the RL policy of any one of the buildings to other buildings provides comparable performance while reducing the cost of training, 3) training RL-controllers on limited temporal data that does not capture full seasonality in occupant behaviour has little effect on performance. Although, the zero-net-energy (ZNE) condition of the buildings could be maintained or worsened as a result of controlled batteries, KPIs that are typically improved by ZNE condition (electricity price and carbon emissions) are further improved when the batteries are managed by an advanced controller.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 11 of its 62 references that are in the map. First posted on arXiv 31 Dec 2022.

Builds on

11 of 62 references.

References · 11

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  3. 2019Challenges of Real-World Reinforcement Learningcited by 291
  4. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learningcited by 63
  5. 2020Physics-constrained Deep Learning of Multi-zone Building Thermal Dynamicscited by 32
  6. 2020CityLearn: Standardizing Research in Multi-Agent Reinforcement Learning for Demand Response and Urban Energy Managementcited by 27
  7. 2021GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Managementcited by 12
  8. 2021Real-world challenges for multi-agent reinforcement learning in grid-interactive buildingscited by 9
  9. 2015Experimental analysis of data-driven control for a building heating systemcited by 8
  10. 2020A Centralised Soft Actor Critic Deep Reinforcement Learning Approach to District Demand Side Management through CityLearncited by 7
  11. 2015Reinforcement Learning Applied to an Electric Water Heater: From Theory to Practicecited by 5

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3.

4 papers.

  1. 2024CityLearn v2: Energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communitiescited by 15
  2. 2023Real-World Implementation of Reinforcement Learning Based Energy Coordination for a Cluster of Householdscited by 1
  3. 2024Decentralized Coordination of Distributed Energy Resources through Local Energy Markets and Deep Reinforcement Learningcited by 0
  4. 2024Generalising Battery Control in Net-Zero Buildings via Personalised Federated RLcited by 0