# Experimental analysis of data-driven control for a building heating system

arXiv: https://arxiv.org/abs/1507.03638 · HTML: https://mlsanity.com/paper/1507.03638/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2015.

## Built on by (most cited first, 8 of 8)

- [MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearn](https://mlsanity.com/paper/2301.01148/) (2022)
- [An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control](https://mlsanity.com/paper/2401.05737/) (2024)
- [Lessons learned from field demonstrations of model predictive control and reinforcement learning for residential and commercial HVAC: A review](https://mlsanity.com/paper/2503.05022/) (2025)
- [Development of a Soft Actor Critic Deep Reinforcement Learning Approach for Harnessing Energy Flexibility in a Large Office Building](https://mlsanity.com/paper/2104.12125/) (2021)
- [Employing Federated Learning for Training Autonomous HVAC Systems](https://mlsanity.com/paper/2405.00389/) (2024)
- [Real-world validation of safe reinforcement learning, model predictive control and decision tree-based home energy management systems](https://mlsanity.com/paper/2408.07435/) (2024)
- [Experimental evaluation of offline reinforcement learning for HVAC control in buildings](https://mlsanity.com/paper/2408.07986/) (2024)
- [Capacity-constrained demand response in smart grids using deep reinforcement learning](https://mlsanity.com/paper/2602.16525/) (2026)

## Abstract (arXiv)

Driven by the opportunity to harvest the flexibility related to building climate control for demand response applications, this work presents a data-driven control approach building upon recent advancements in reinforcement learning. More specifically, model assisted batch reinforcement learning is applied to the setting of building climate control subjected to a dynamic pricing. The underlying sequential decision making problem is cast on a markov decision problem, after which the control algorithm is detailed. In this work, fitted Q-iteration is used to construct a policy from a batch of experimental tuples. In those regions of the state space where the experimental sample density is low, virtual support samples are added using an artificial neural network. Finally, the resulting policy is shaped using domain knowledge. The control approach has been evaluated quantitatively using a simulation and qualitatively in a living lab. From the quantitative analysis it has been found that the control approach converges in approximately 20 days to obtain a control policy with a performance within 90% of the mathematical optimum. The experimental analysis confirms that within 10 to 20 days sensible policies are obtained that can be used for different outside temperature regimes.