# Demand response for residential building heating: Effective Monte Carlo Tree Search control based on physics-informed neural networks

arXiv: https://arxiv.org/abs/2312.03365 · HTML: https://mlsanity.com/paper/2312.03365/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 60 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 9 of 9)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Physics-constrained Deep Learning of Multi-zone Building Thermal Dynamics](https://mlsanity.com/paper/2011.05987/) (2020)
- [Physics Informed Neural Networks for Control Oriented Thermal Modeling of Buildings](https://mlsanity.com/paper/2111.12066/) (2021)
- [Physically Consistent Neural Networks for building thermal modeling: theory and analysis](https://mlsanity.com/paper/2112.03212/) (2021)
- [Deep Reinforcement Learning for Optimal Control of Space Heating](https://mlsanity.com/paper/1805.03777/) (2018)
- [Experimental Demonstration of Frequency Regulation by Commercial Buildings - Part I: Modeling and Hierarchical Control Design](https://mlsanity.com/paper/1605.05835/) (2016)

## Built on by (most cited first, 4 of 4)

- [Model Predictive Control-Guided Reinforcement Learning for Implicit Balancing](https://mlsanity.com/paper/2510.04868/) (2025)
- [Integrating Physics-Based and Data-Driven Approaches for Probabilistic Building Energy Modeling](https://mlsanity.com/paper/2507.17526/) (2025)
- [Partially stochastic deep learning with uncertainty quantification for model predictive heating control](https://mlsanity.com/paper/2504.03350/) (2025)
- [S3TS: Stochastic Scenario-Structured Tree Search for Advanced Planning Under Uncertainty](https://mlsanity.com/paper/2606.02151/) (2026)

## Abstract (arXiv)

To reduce global carbon emissions and limit climate change, controlling energy consumption in buildings is an important piece of the puzzle. Here, we specifically focus on using a demand response (DR) algorithm to limit the energy consumption of a residential building's heating system while respecting user's thermal comfort. In that domain, Reinforcement learning (RL) methods have been shown to be quite effective. One such RL method is Monte Carlo Tree Search (MCTS), which has achieved impressive success in playing board games (go, chess). A particular advantage of MCTS is that its decision tree structure naturally allows to integrate exogenous constraints (e.g., by trimming branches that violate them), while conventional RL solutions need more elaborate techniques (e.g., indirectly by adding penalties in the cost/reward function, or through a backup controller that corrects constraint-violating actions). The main aim of this paper is to study the adoption of MCTS for building control, since this (to the best of our knowledge) has remained largely unexplored. A specific property of MCTS is that it needs a simulator component that can predict subsequent system states, based on actions taken. A straightforward data-driven solution is to use black-box neural networks (NNs). We will however extend a Physics-informed Neural Network (PiNN) model to deliver multi-timestep predictions, and show the benefit it offers in terms of lower prediction errors (-32% MAE) as well as better MCTS performance (-4% energy cost, +7% thermal comfort) compared to a black-box NN. A second contribution will be to extend a vanilla MCTS version to adopt the ideas applied in AlphaZero (i.e., using learned prior and value functions and an action selection heuristic) to obtain lower computational costs while maintaining control performance.