# Building HVAC Scheduling Using Reinforcement Learning via Neural Network Based Model Approximation

arXiv: https://arxiv.org/abs/1910.05313 · HTML: https://mlsanity.com/paper/1910.05313/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2019.

## Built on by (most cited first, 16 of 16)

- [BRAC+: Improved Behavior Regularized Actor Critic for Offline Reinforcement Learning](https://mlsanity.com/paper/2110.00894/) (2021)
- [Generative AI in the Construction Industry: Opportunities & Challenges](https://mlsanity.com/paper/2310.04427/) (2023)
- [A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings](https://mlsanity.com/paper/2310.08569/) (2023)
- [Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control](https://mlsanity.com/paper/2403.00172/) (2024)
- [A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control](https://mlsanity.com/paper/2407.12195/) (2024)
- [The Smart Buildings Control Suite: A Diverse Open Source Benchmark to Evaluate and Scale HVAC Control Policies for Sustainability](https://mlsanity.com/paper/2410.03756/) (2024)
- [An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control](https://mlsanity.com/paper/2401.05737/) (2024)
- [Low Emission Building Control with Zero-Shot Reinforcement Learning](https://mlsanity.com/paper/2208.06385/) (2022)
- [Safe Reinforcement Learning for Power System Control: A Review](https://mlsanity.com/paper/2407.00681/) (2024)
- [Parallel Actors and Learners: A Framework for Generating Scalable RL Implementations](https://mlsanity.com/paper/2110.01101/) (2021)
- [Physics-Informed Data Denoising for Real-Life Sensing Systems](https://mlsanity.com/paper/2311.06968/) (2023)
- [B2RL: An open-source Dataset for Building Batch Reinforcement Learning](https://mlsanity.com/paper/2209.15626/) (2022)
- [Low Emission Building Control with Zero-Shot Reinforcement Learning](https://mlsanity.com/paper/2206.14191/) (2022)
- [Experimental evaluation of offline reinforcement learning for HVAC control in buildings](https://mlsanity.com/paper/2408.07986/) (2024)
- [MuFlex: A Scalable, Physics-based Platform for Multi-Building Flexibility Analysis and Coordination](https://mlsanity.com/paper/2508.13532/) (2025)
- [Software Engineering for AI-driven Building Operation](https://mlsanity.com/paper/2608.16237/) (2026)

## Abstract (arXiv)

Buildings sector is one of the major consumers of energy in the United States. The buildings HVAC (Heating, Ventilation, and Air Conditioning) systems, whose functionality is to maintain thermal comfort and indoor air quality (IAQ), account for almost half of the energy consumed by the buildings. Thus, intelligent scheduling of the building HVAC system has the potential for tremendous energy and cost savings while ensuring that the control objectives (thermal comfort, air quality) are satisfied. Recently, several works have focused on model-free deep reinforcement learning based techniques such as Deep Q-Network (DQN). Such methods require extensive interactions with the environment. Thus, they are impractical to implement in real systems due to low sample efficiency. Safety-aware exploration is another challenge in real systems since certain actions at particular states may result in catastrophic outcomes. To address these issues and challenges, we propose a model-based reinforcement learning approach that learns the system dynamics using a neural network. Then, we adopt Model Predictive Control (MPC) using the learned system dynamics to perform control with random-sampling shooting method. To ensure safe exploration, we limit the actions within safe range and the maximum absolute change of actions according to prior knowledge. We evaluate our ideas through simulation using widely adopted EnergyPlus tool on a case study consisting of a two zone data-center. Experiments show that the average deviation of the trajectories sampled from the learned dynamics and the ground truth is below 20%. Compared with baseline approaches, we reduce the total energy consumption by 17.1% ~ 21.8%. Compared with model-free reinforcement learning approach, we reduce the required number of training steps to converge by 10x.