# Go Beyond Black-box Policies: Rethinking the Design of Learning Agent for Interpretable and Verifiable HVAC Control

arXiv: https://arxiv.org/abs/2403.00172 · HTML: https://mlsanity.com/paper/2403.00172/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 7 of its 17 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 7 of 7)

- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [Challenges in Deploying Machine Learning: a Survey of Case Studies](https://mlsanity.com/paper/2011.09926/) (2020)
- [On a generalization of the Jensen-Shannon divergence and the JS-symmetrization of distances relying on abstract means](https://mlsanity.com/paper/1904.04017/) (2019)
- [Building HVAC Scheduling Using Reinforcement Learning via Neural Network Based Model Approximation](https://mlsanity.com/paper/1910.05313/) (2019)
- [Reward Bound for Behavioral Guarantee of Model-based Planning Agents](https://mlsanity.com/paper/2402.13419/) (2024)

## Built on by (most cited first, 5 of 5)

- [A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control](https://mlsanity.com/paper/2407.12195/) (2024)
- [MARLP: Time-series Forecasting Control for Agricultural Managed Aquifer Recharge](https://mlsanity.com/paper/2407.01005/) (2024)
- [Towards VM Rescheduling Optimization Through Deep Reinforcement Learning](https://mlsanity.com/paper/2505.17359/) (2025)
- [Exact Action Values Are Not Enough: Rollout-Verified Reinforcement Fine-Tuning of a Reasoning Model for Multi-Zone VAV Control](https://mlsanity.com/paper/2607.27914/) (2026)
- [Heating in human-HVAC interaction for smart homes: An interdisciplinary overview](https://mlsanity.com/paper/2609.23508/) (2026)

## Abstract (arXiv)

Recent research has shown the potential of Model-based Reinforcement Learning (MBRL) to enhance energy efficiency of Heating, Ventilation, and Air Conditioning (HVAC) systems. However, existing methods rely on black-box thermal dynamics models and stochastic optimizers, lacking reliability guarantees and posing risks to occupant health. In this work, we overcome the reliability bottleneck by redesigning HVAC controllers using decision trees extracted from existing thermal dynamics models and historical data. Our decision tree-based policies are deterministic, verifiable, interpretable, and more energy-efficient than current MBRL methods. First, we introduce a novel verification criterion for RL agents in HVAC control based on domain knowledge. Second, we develop a policy extraction procedure that produces a verifiable decision tree policy. We found that the high dimensionality of the thermal dynamics model input hinders the efficiency of policy extraction. To tackle the dimensionality challenge, we leverage importance sampling conditioned on historical data distributions, significantly improving policy extraction efficiency. Lastly, we present an offline verification algorithm that guarantees the reliability of a control policy. Extensive experiments show that our method saves 68.4% more energy and increases human comfort gain by 14.8% compared to the state-of-the-art method, in addition to an 1127x reduction in computation overhead. Our code and data are available at https://github.com/ryeii/Veri_HVAC