# Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

arXiv: https://arxiv.org/abs/2006.14156 · HTML: https://mlsanity.com/paper/2006.14156/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jun 2020.

## Built on by (most cited first, 26 of 26)

- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [AI for Global Climate Cooperation: Modeling Global Climate Negotiations, Agreements, and Long-Term Cooperation in RICE-N](https://mlsanity.com/paper/2208.07004/) (2022)
- [Pre-Trained Large Language Models for Industrial Control](https://mlsanity.com/paper/2308.03028/) (2023)
- [A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings](https://mlsanity.com/paper/2310.08569/) (2023)
- [The Smart Buildings Control Suite: A Diverse Open Source Benchmark to Evaluate and Scale HVAC Control Policies for Sustainability](https://mlsanity.com/paper/2410.03756/) (2024)
- [A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control](https://mlsanity.com/paper/2308.05711/) (2023)
- [Active Reinforcement Learning for Robust Building Control](https://mlsanity.com/paper/2312.10289/) (2023)
- [An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control](https://mlsanity.com/paper/2401.05737/) (2024)
- [CommonPower: A Framework for Safe Data-Driven Smart Grid Control](https://mlsanity.com/paper/2406.03231/) (2024)
- [Physics-informed Evolutionary Strategy based Control for Mitigating Delayed Voltage Recovery](https://mlsanity.com/paper/2111.14352/) (2021)
- [Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning](https://mlsanity.com/paper/2203.12048/) (2022)
- [Reinforcement Learning for Sustainable Energy: A Survey](https://mlsanity.com/paper/2407.18597/) (2024)
- [Machine Learning for Smart and Energy-Efficient Buildings](https://mlsanity.com/paper/2211.14889/) (2022)
- [Energy Optimization for HVAC Systems in Multi-VAV Open Offices: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2306.13333/) (2023)
- [Decentralized Diffusion Policy Learning for Enhanced Exploration in Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2605.07101/) (2026)
- [Scalable Multi-Agent Reinforcement Learning for Residential Load Scheduling under Data Governance](https://mlsanity.com/paper/2110.02784/) (2021)
- [TreeC: a method to generate interpretable energy management systems using a metaheuristic algorithm](https://mlsanity.com/paper/2304.08310/) (2023)
- [Laxity-Aware Scalable Reinforcement Learning for HVAC Control](https://mlsanity.com/paper/2306.16619/) (2023)
- [Learning a Reward Function for User-Preferred Appliance Scheduling](https://mlsanity.com/paper/2310.07389/) (2023)
- [A Multi-Agent Reinforcement Learning Framework for Public Health Decision Analysis](https://mlsanity.com/paper/2311.00855/) (2023)
- [A Distributed ADMM-based Deep Learning Approach for Thermal Control in Multi-Zone Buildings under Demand Response Events](https://mlsanity.com/paper/2312.05073/) (2023)
- [Experimental evaluation of offline reinforcement learning for HVAC control in buildings](https://mlsanity.com/paper/2408.07986/) (2024)
- [STEMS: Spatial-Temporal Enhanced Safe Multi-Agent Coordination for Building Energy Management](https://mlsanity.com/paper/2510.14112/) (2025)
- [EcoNet: Multiagent Planning and Control Of Household Energy Resources Using Active Inference](https://mlsanity.com/paper/2512.21343/) (2025)
- [The Smart Buildings Challenge: Learning Foundation Models for Real-World Optimization at Scale](https://mlsanity.com/paper/2610.18511/) (2026)
- [One Agent per Zone: Decentralised Active Inference under Partial Observability for Multi-Zone HVAC Control on Edge Hardware](https://mlsanity.com/paper/2610.19790/) (2026)

## Abstract (arXiv)

In commercial buildings, about 40%-50% of the total electricity consumption is attributed to Heating, Ventilation, and Air Conditioning (HVAC) systems, which places an economic burden on building operators. In this paper, we intend to minimize the energy cost of an HVAC system in a multi-zone commercial building under dynamic pricing with the consideration of random zone occupancy, thermal comfort, and indoor air quality comfort. Due to the existence of unknown thermal dynamics models, parameter uncertainties (e.g., outdoor temperature, electricity price, and number of occupants), spatially and temporally coupled constraints associated with indoor temperature and CO2 concentration, a large discrete solution space, and a non-convex and non-separable objective function, it is very challenging to achieve the above aim. To this end, the above energy cost minimization problem is reformulated as a Markov game. Then, an HVAC control algorithm is proposed to solve the Markov game based on multi-agent deep reinforcement learning with attention mechanism. The proposed algorithm does not require any prior knowledge of uncertain parameters and can operate without knowing building thermal dynamics models. Simulation results based on real-world traces show the effectiveness, robustness and scalability of the proposed algorithm.