# A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control

arXiv: https://arxiv.org/abs/2308.05711 · HTML: https://mlsanity.com/paper/2308.05711/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 26 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 6 of 6)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings](https://mlsanity.com/paper/2006.14156/) (2020)
- [A Review of Reinforcement Learning for Autonomous Building Energy Management](https://mlsanity.com/paper/1903.05196/) (2019)
- [Reinforcement Learning Testbed for Power-Consumption Optimization](https://mlsanity.com/paper/1808.10427/) (2018)

## Built on by (most cited first, 4 of 4)

- [A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings](https://mlsanity.com/paper/2310.08569/) (2023)
- [The Smart Buildings Control Suite: A Diverse Open Source Benchmark to Evaluate and Scale HVAC Control Policies for Sustainability](https://mlsanity.com/paper/2410.03756/) (2024)
- [ThermoLLM: Thermodynamics-Aware HVAC Control with Spatial-Semantic Knowledge Graph](https://mlsanity.com/paper/2606.22911/) (2026)
- [The Smart Buildings Challenge: Learning Foundation Models for Real-World Optimization at Scale](https://mlsanity.com/paper/2610.18511/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) is a promising approach for optimizing HVAC control. RL offers a framework for improving system performance, reducing energy consumption, and enhancing cost efficiency. We benchmark two popular classical and deep RL methods (Q-Learning and Deep-Q-Networks) across multiple HVAC environments and explore the practical consideration of model hyper-parameter selection and reward tuning. The findings provide insight for configuring RL agents in HVAC systems, promoting energy-efficient and cost-effective operation.