# Real-world validation of safe reinforcement learning, model predictive control and decision tree-based home energy management systems

arXiv: https://arxiv.org/abs/2408.07435 · HTML: https://mlsanity.com/paper/2408.07435/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 7 of its 49 references that are in the map.
- First posted on arXiv 14 Aug 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 7 of 7)

- [Model-predictive control and reinforcement learning in multi-energy system case studies](https://mlsanity.com/paper/2104.09785/) (2021)
- [Real-world challenges for multi-agent reinforcement learning in grid-interactive buildings](https://mlsanity.com/paper/2112.06127/) (2021)
- [Experimental analysis of data-driven control for a building heating system](https://mlsanity.com/paper/1507.03638/) (2015)
- [Safe reinforcement learning for multi-energy management systems with known constraint functions](https://mlsanity.com/paper/2207.03830/) (2022)
- [Deep Reinforcement Learning based Optimal Control of Hot Water Systems](https://mlsanity.com/paper/1801.01467/) (2018)
- [An adaptive safety layer with hard constraints for safe reinforcement learning in multi-energy management systems](https://mlsanity.com/paper/2304.08897/) (2023)
- [Real-World Implementation of Reinforcement Learning Based Energy Coordination for a Cluster of Households](https://mlsanity.com/paper/2310.19155/) (2023)

## Abstract (arXiv)

Recent advancements in machine learning based energy management approaches, specifically reinforcement learning with a safety layer (OptLayerPolicy) and a metaheuristic algorithm generating a decision tree control policy (TreeC), have shown promise. However, their effectiveness has only been demonstrated in computer simulations. This paper presents the real-world validation of these methods, comparing against model predictive control and simple rule-based control benchmark. The experiments were conducted on the electrical installation of 4 reproductions of residential houses, which all have their own battery, photovoltaic and dynamic load system emulating a non-controllable electrical load and a controllable electric vehicle charger. The results show that the simple rules, TreeC, and model predictive control-based methods achieved similar costs, with a difference of only 0.6%. The reinforcement learning based method, still in its training phase, obtained a cost 25.5% higher to the other methods. Additional simulations show that the costs can be further reduced by using a more representative training dataset for TreeC and addressing errors in the model predictive control implementation caused by its reliance on accurate data from various sources. The OptLayerPolicy safety layer allows safe online training of a reinforcement learning agent in the real-world, given an accurate constraint function formulation. The proposed safety layer method remains error-prone, nonetheless, it is found beneficial for all investigated methods. The TreeC method, which does require building a realistic simulation for training, exhibits the safest operational performance, exceeding the grid limit by only 27.1 Wh compared to 593.9 Wh for reinforcement learning.