# Laxity-Aware Scalable Reinforcement Learning for HVAC Control

arXiv: https://arxiv.org/abs/2306.16619 · HTML: https://mlsanity.com/paper/2306.16619/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [On-line Building Energy Optimization using Deep Reinforcement Learning](https://mlsanity.com/paper/1707.05878/) (2017)
- [Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings](https://mlsanity.com/paper/2006.14156/) (2020)
- [Learning-Based Predictive Control via Real-Time Aggregate Flexibility](https://mlsanity.com/paper/2012.11261/) (2020)
- [Smoothed Least-Laxity-First Algorithm for EV Charging](https://mlsanity.com/paper/2102.08610/) (2021)
- [Modeling and Optimization of Complex Building Energy Systems with Deep Neural Networks](https://mlsanity.com/paper/1711.02278/) (2017)
- [Online Learning and Distributed Control for Residential Demand Response](https://mlsanity.com/paper/2010.05153/) (2020)
- [End-to-End Demand Response Model Identification and Baseline Estimation with Deep Learning](https://mlsanity.com/paper/2109.00741/) (2021)
- [Learning a Multi-Agent Controller for Shared Energy Storage System](https://mlsanity.com/paper/2302.08328/) (2023)
- [A Geometric Approach to Aggregate Flexibility Modeling of Thermostatically Controlled Loads](https://mlsanity.com/paper/1608.04422/) (2016)
- [Data-Driven Demand-Side Flexibility Quantification: Prediction and Approximation of Flexibility Envelopes](https://mlsanity.com/paper/2110.12796/) (2021)
- [Experimental implementation of an emission-aware prosumer with online flexibility quantification and provision](https://mlsanity.com/paper/2110.12831/) (2021)
- [Efficient Representation for Electric Vehicle Charging Station Operations using Reinforcement Learning](https://mlsanity.com/paper/2108.03236/) (2021)

## Abstract (arXiv)

Demand flexibility plays a vital role in maintaining grid balance, reducing peak demand, and saving customers' energy bills. Given their highly shiftable load and significant contribution to a building's energy consumption, Heating, Ventilation, and Air Conditioning (HVAC) systems can provide valuable demand flexibility to the power systems by adjusting their energy consumption in response to electricity price and power system needs. To exploit this flexibility in both operation time and power, it is imperative to accurately model and aggregate the load flexibility of a large population of HVAC systems as well as designing effective control algorithms. In this paper, we tackle the curse of dimensionality issue in modeling and control by utilizing the concept of laxity to quantify the emergency level of each HVAC operation request. We further propose a two-level approach to address energy optimization for a large population of HVAC systems. The lower level involves an aggregator to aggregate HVAC load laxity information and use least-laxity-first (LLF) rule to allocate real-time power for individual HVAC systems based on the controller's total power. Due to the complex and uncertain nature of HVAC systems, we leverage a reinforcement learning (RL)-based controller to schedule the total power based on the aggregated laxity information and electricity price. We evaluate the temperature control and energy cost saving performance of a large-scale group of HVAC systems in both single-zone and multi-zone scenarios, under varying climate and electricity market conditions. The experiment results indicate that proposed approach outperforms the centralized methods in the majority of test scenarios, and performs comparably to model-based method in some scenarios.