# Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control

arXiv: https://arxiv.org/abs/2110.13450 · HTML: https://mlsanity.com/paper/2110.13450/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Distributed Prioritized Experience Replay](https://mlsanity.com/paper/1803.00933/) (2018)
- [Optimal Control Via Neural Networks: A Convex Approach](https://mlsanity.com/paper/1805.11835/) (2018)
- [Reinforcement Learning Testbed for Power-Consumption Optimization](https://mlsanity.com/paper/1808.10427/) (2018)
- [Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning](https://mlsanity.com/paper/1901.04693/) (2019)

## Built on by (most cited first, 5 of 5)

- [Semi-analytical Industrial Cooling System Model for Reinforcement Learning](https://mlsanity.com/paper/2207.13131/) (2022)
- [An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control](https://mlsanity.com/paper/2401.05737/) (2024)
- [Optimizing Industrial HVAC Systems with Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2209.08112/) (2022)
- [Machine Learning for Smart and Energy-Efficient Buildings](https://mlsanity.com/paper/2211.14889/) (2022)
- [Identifying Culprits Through Deep Deterministic Policy Gradient Deep Learning Investigation](https://mlsanity.com/paper/2605.14774/) (2026)

## Abstract (arXiv)

It is estimated that about 40%-50% of total electricity consumption in commercial buildings can be attributed to Heating, Ventilation, and Air Conditioning (HVAC) systems. Minimizing the energy cost while considering the thermal comfort of the occupants is very challenging due to unknown and complex relationships between various HVAC controls and thermal dynamics inside a building. To this end, we present a multi-agent, distributed deep reinforcement learning (DRL) framework based on Energy Plus simulation environment for optimizing HVAC in commercial buildings. This framework learns the complex thermal dynamics in the building and takes advantage of the differential effect of cooling and heating systems in the building to reduce energy costs, while maintaining the thermal comfort of the occupants. With adaptive penalty, the RL algorithm can be prioritized for energy savings or maintaining thermal comfort. Using DRL, we achieve more than 75% savings in energy consumption. The distributed DRL framework can be scaled to multiple GPUs and CPUs of heterogeneous types.