# Deep Reinforcement Learning for Heat Pump Control

arXiv: https://arxiv.org/abs/2212.12716 · HTML: https://mlsanity.com/paper/2212.12716/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 24 references that are in the map.
- First posted on arXiv 24 Dec 2022.

## Builds on (most cited first, 6 of 6)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Deep Reinforcement Learning for Optimal Control of Space Heating](https://mlsanity.com/paper/1805.03777/) (2018)
- [Residential Demand Response Applications Using Batch Reinforcement Learning](https://mlsanity.com/paper/1504.02125/) (2015)

## Built on by (most cited first, 3 of 3)

- [Model-based deep reinforcement learning for accelerated learning from flow simulations](https://mlsanity.com/paper/2402.16543/) (2024)
- [Constrained Reinforcement Learning for Safe Heat Pump Control](https://mlsanity.com/paper/2409.19716/) (2024)
- [Deadline-Aware, Energy-Efficient Control of Domestic Immersion Hot Water Heater](https://mlsanity.com/paper/2601.18123/) (2026)

## Abstract (arXiv)

Heating in private households is a major contributor to the emissions generated today. Heat pumps are a promising alternative for heat generation and are a key technology in achieving our goals of the German energy transformation and to become less dependent on fossil fuels. Today, the majority of heat pumps in the field are controlled by a simple heating curve, which is a naive mapping of the current outdoor temperature to a control action. A more advanced control approach is model predictive control (MPC) which was applied in multiple research works to heat pump control. However, MPC is heavily dependent on the building model, which has several disadvantages. Motivated by this and by recent breakthroughs in the field, this work applies deep reinforcement learning (DRL) to heat pump control in a simulated environment. Through a comparison to MPC, it could be shown that it is possible to apply DRL in a model-free manner to achieve MPC-like performance. This work extends other works which have already applied DRL to building heating operation by performing an in-depth analysis of the learned control strategies and by giving a detailed comparison of the two state-of-the-art control methods.