# Comparative Field Deployment of Reinforcement Learning and Model Predictive Control for Residential HVAC

arXiv: https://arxiv.org/abs/2510.01475 · HTML: https://mlsanity.com/paper/2510.01475/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 16 of its 77 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 16 of 16)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning](https://mlsanity.com/paper/2303.05479/) (2023)
- [Differentiable MPC for End-to-end Planning and Control](https://mlsanity.com/paper/1810.13400/) (2018)
- [Controlling Commercial Cooling Systems Using Reinforcement Learning](https://mlsanity.com/paper/2211.07357/) (2022)
- [Deep Learning Alternative to Explicit Model Predictive Control for Unknown Nonlinear Systems](https://mlsanity.com/paper/2011.03699/) (2020)
- [Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.05077/) (2017)
- [Physics-informed linear regression is competitive with two Machine Learning methods in residential building MPC](https://mlsanity.com/paper/2110.15911/) (2021)
- [Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: simulations and experiments](https://mlsanity.com/paper/2103.01886/) (2021)
- [Lessons learned from field demonstrations of model predictive control and reinforcement learning for residential and commercial HVAC: A review](https://mlsanity.com/paper/2503.05022/) (2025)
- [Long-term experimental study of price responsive predictive control in a real occupied single-family house with heat pump](https://mlsanity.com/paper/2303.16289/) (2023)
- [Field demonstration of predictive heating control for an all-electric house in a cold climate](https://mlsanity.com/paper/2402.07032/) (2024)
- [Humidity-Aware Model Predictive Control for Residential Air Conditioning: A Field Study](https://mlsanity.com/paper/2407.01707/) (2024)
- [Protecting residential electrical panels and service through model predictive control: A field study](https://mlsanity.com/paper/2409.04884/) (2024)
- [Why has advanced commercial HVAC control not yet achieved its promise?](https://mlsanity.com/paper/2411.06204/) (2024)
- [Unmasking the Role of Remote Sensors in Comfort, Energy and Demand Response](https://mlsanity.com/paper/2404.15368/) (2024)

## Abstract (arXiv)

Model Predictive Control (MPC) has demonstrated significant performance improvements over today's control methods for residential Heating, Ventilation, and Air Conditioning (HVAC), but deploying MPC often requires substantial engineering effort. Reinforcement Learning (RL) may offer comparable performance with easier deployment, but its practical application for residential HVAC remains largely undemonstrated, leaving open questions related to occupant comfort and data requirements. To investigate these issues, we deployed one MPC variant and one model-based RL variant for one month each in an occupied house in a cold climate. The controllers adjusted an air-to-air heat pump's thermostat temperature setpoint based on measurements of the indoor temperature and the electric power used for heating. Relative to constant-setpoint operation, MPC saved 18.1% (95% confidence interval: 4.4 to 30.9%) of weather-normalized heat pump energy and RL saved 20.9% (2.6 to 38.3%). MPC maintained acceptable occupant comfort. RL kept the house cooler, particularly during an initial adaptation phase, leading to three reports of occupant discomfort. The two algorithms had similar data requirements. We estimate that for a fresh deployment in another house, RL would take about one-third less engineering effort than MPC. While RL reduces deployment effort, it faces difficulties related to safe controller initialization and to mismatches between the modeled and true state and action spaces.