# Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: simulations and experiments

arXiv: https://arxiv.org/abs/2103.01886 · HTML: https://mlsanity.com/paper/2103.01886/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 85 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Brief Survey of Deep Reinforcement Learning](https://mlsanity.com/paper/1708.05866/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Recent Trends in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/1708.02709/) (2017)
- [A Critical Review of Recurrent Neural Networks for Sequence Learning](https://mlsanity.com/paper/1506.00019/) (2015)
- [Optimal Control Via Neural Networks: A Convex Approach](https://mlsanity.com/paper/1805.11835/) (2018)
- [On-line Building Energy Optimization using Deep Reinforcement Learning](https://mlsanity.com/paper/1707.05878/) (2017)
- [A Review of Reinforcement Learning for Autonomous Building Energy Management](https://mlsanity.com/paper/1903.05196/) (2019)
- [One for Many: Transfer Learning for Building HVAC Control](https://mlsanity.com/paper/2008.03625/) (2020)
- [Definition and evaluation of model-free coordination of electrical vehicle charging with reinforcement learning](https://mlsanity.com/paper/1809.10679/) (2018)
- [Combining Deep Reinforcement Learning and Safety Based Control for Autonomous Driving](https://mlsanity.com/paper/1612.00147/) (2016)
- [Performance-Driven Cascade Controller Tuning with Bayesian Optimization](https://mlsanity.com/paper/2007.12536/) (2020)
- [Modeling and Optimization of Complex Building Energy Systems with Deep Neural Networks](https://mlsanity.com/paper/1711.02278/) (2017)
- [A coordinated control to improve performance for a building cluster with energy storage, electric vehicles, and energy sharing considered](https://mlsanity.com/paper/2102.09793/) (2021)

## Built on by (most cited first, 8 of 8)

- [Physically Consistent Neural Networks for building thermal modeling: theory and analysis](https://mlsanity.com/paper/2112.03212/) (2021)
- [Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control](https://mlsanity.com/paper/2203.05434/) (2022)
- [Towards Scalable Physically Consistent Neural Networks: an Application to Data-driven Multi-zone Thermal Building Models](https://mlsanity.com/paper/2212.12380/) (2022)
- [Data-driven adaptive building thermal controller tuning with constraints: A primal-dual contextual Bayesian optimization approach](https://mlsanity.com/paper/2310.00758/) (2023)
- [Lessons learned from field demonstrations of model predictive control and reinforcement learning for residential and commercial HVAC: A review](https://mlsanity.com/paper/2503.05022/) (2025)
- [Lessons Learned from Data-Driven Building Control Experiments: Contrasting Gaussian Process-based MPC, Bilevel DeePC, and Deep Reinforcement Learning](https://mlsanity.com/paper/2205.15703/) (2022)
- [Path Planning using Reinforcement Learning: A Policy Iteration Approach](https://mlsanity.com/paper/2303.07535/) (2023)
- [Comparative Field Deployment of Reinforcement Learning and Model Predictive Control for Residential HVAC](https://mlsanity.com/paper/2510.01475/) (2025)

## Abstract (arXiv)

This work presents a fully data-driven, black-box pipeline to obtain an optimal control policy for a multi-loop building control problem based on historical building and weather data, thus without the need for complex physics-based modelling. We demonstrate the method for joint control of room temperature and bidirectional EV charging to maximize the occupant thermal comfort and energy savings while leaving enough energy in the EV battery for the next trip. We modelled the room temperature with a recurrent neural network and EV charging with a piece-wise linear function. Using these models as a simulation environment, we applied a deep reinforcement learning (DRL) algorithm to obtain an optimal control policy. The learnt policy achieves on average 17% energy savings over the heating season and 19% better comfort satisfaction than a standard RB room temperature controller. When a bidirectional EV is additionally connected and a two-tariff electricity pricing is applied, the MIMO DRL policy successfully leverages the battery and decreases the overall cost of electricity compared to two standard RB controllers, one controlling the room temperature and another controlling the bidirectional EV (dis-)charging. Finally, we demonstrate a successful transfer of the learnt DRL policy from simulation onto a real building, the DFAB HOUSE at Empa Duebendorf in Switzerland, achieving up to 30% energy savings while maintaining similar comfort levels compared to a conventional RB room temperature controller over three weeks during the heating season.