mlsanityData to 14 Oct 2026

Safe Deep Reinforcement Learning for Energy-Efficient HVAC Control in Multi-Zone Residential Buildings

arXiv 2608.17235 ↗First posted 18 Aug 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

HVAC systems represent a major share of building energy consumption. Traditional control strategies are limited in coordinating energy-comfort tradeoffs across multiple zones simultaneously. Reinforcement learning (RL) offers adaptive, data-driven control that optimizes performance over time. However, deploying learned neural network controllers in safety-critical building systems remains challenging due to lack of formal safety guarantees. We propose a safety-certified deep RL framework for multi-zone residential HVAC control. Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC) agents are trained in an EnergyPlus/Sinergym simulation to minimize energy consumption while maintaining thermal comfort. Post-training safety certification is performed on the PPO policy using Lipschitz-based forward invariance analysis, building on existing tools for the computation of Lipschitz constants for neural networks, to guarantee constraint satisfaction. Both agents are evaluated over an annual simulation cycle in an eight-zone variable refrigerant flow (VRF) testbed. The PPO agent achieves 67% comfort violation reduction compared to rule-based control, while the SAC agent achieves 27.6% energy savings. The PPO policy satisfies formal safety certification with a margin of 2.003^circC. These results demonstrate the feasibility of combining reinforcement learning with post-training safety verification for multi-zone building control.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 26 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

10 of 26 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learning11 Dec
  2. 2024An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control11 Jan

Other references · 8

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  6. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 293
  7. 2022Data-Driven Abstraction-Based Control Synthesiscited by 8
  8. 2023Safety verification of Neural-Network-based controllers: a set invariance approachcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Safe Deep Reinforcement Learning for Building Heating Control and Demand-side Flexibility
  2. 2026Hierarchical Control Framework Integrating LLMs with RL for Decarbonized HVAC Operation
  3. 2024A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control
  4. 2024HVAC-DPT: A Decision Pretrained Transformer for HVAC Control
  5. 2026Generalizing HVAC Control With Domain Randomized Reinforcement Learning
  6. 2024Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control
  7. 2025Continual Reinforcement Learning for HVAC Systems Control: Integrating Hypernetworks and Transfer Learning
  8. 2026Exact Action Values Are Not Enough: Rollout-Verified Reinforcement Fine-Tuning of a Reasoning Model for Multi-Zone VAV Control