# An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control

arXiv: https://arxiv.org/abs/2401.05737 · HTML: https://mlsanity.com/paper/2401.05737/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 100 references that are in the map.
- First posted on arXiv 11 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Automatic Curriculum Learning For Deep RL: A Short Survey](https://mlsanity.com/paper/2003.04664/) (2020)
- [Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control](https://mlsanity.com/paper/1708.04133/) (2017)
- [A Review of Deep Reinforcement Learning for Smart Building Energy Management](https://mlsanity.com/paper/2008.05074/) (2020)
- [On-line Building Energy Optimization using Deep Reinforcement Learning](https://mlsanity.com/paper/1707.05878/) (2017)
- [Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings](https://mlsanity.com/paper/2006.14156/) (2020)
- [A Review of Reinforcement Learning for Autonomous Building Energy Management](https://mlsanity.com/paper/1903.05196/) (2019)
- [Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.05077/) (2017)
- [Reinforcement Learning Testbed for Power-Consumption Optimization](https://mlsanity.com/paper/1808.10427/) (2018)
- [Building HVAC Scheduling Using Reinforcement Learning via Neural Network Based Model Approximation](https://mlsanity.com/paper/1910.05313/) (2019)
- [One for Many: Transfer Learning for Building HVAC Control](https://mlsanity.com/paper/2008.03625/) (2020)
- [Experimental analysis of data-driven control for a building heating system](https://mlsanity.com/paper/1507.03638/) (2015)
- [Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control](https://mlsanity.com/paper/2110.13450/) (2021)

## Built on by (most cited first, 4 of 4)

- [SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learning](https://mlsanity.com/paper/2412.08293/) (2024)
- [ThermoLLM: Thermodynamics-Aware HVAC Control with Spatial-Semantic Knowledge Graph](https://mlsanity.com/paper/2606.22911/) (2026)
- [Physics-enhanced reinforcement learning for real-time optimal control of dynamical systems](https://mlsanity.com/paper/2607.16177/) (2026)
- [Safe Deep Reinforcement Learning for Energy-Efficient HVAC Control in Multi-Zone Residential Buildings](https://mlsanity.com/paper/2608.17235/) (2026)

## Abstract (arXiv)

Heating, Ventilation, and Air Conditioning (HVAC) systems are a major driver of energy consumption in commercial and residential buildings. Recent studies have shown that Deep Reinforcement Learning (DRL) algorithms can outperform traditional reactive controllers. However, DRL-based solutions are generally designed for ad hoc setups and lack standardization for comparison. To fill this gap, this paper provides a critical and reproducible evaluation, in terms of comfort and energy consumption, of several state-of-the-art DRL algorithms for HVAC control. The study examines the controllers' robustness, adaptability, and trade-off between optimization goals by using the Sinergym framework. The results obtained confirm the potential of DRL algorithms, such as SAC and TD3, in complex scenarios and reveal several challenges related to generalization and incremental learning.