# Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control

arXiv: https://arxiv.org/abs/2203.05434 · HTML: https://mlsanity.com/paper/2203.05434/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 10 Mar 2022.

## Builds on (most cited first, 9 of 9)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization](https://mlsanity.com/paper/2105.08881/) (2021)
- [Physically Consistent Neural Networks for building thermal modeling: theory and analysis](https://mlsanity.com/paper/2112.03212/) (2021)
- [Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: simulations and experiments](https://mlsanity.com/paper/2103.01886/) (2021)
- [Deep Reinforcement Learning for Optimal Control of Space Heating](https://mlsanity.com/paper/1805.03777/) (2018)

## Built on by (most cited first, 4 of 4)

- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Towards Scalable Physically Consistent Neural Networks: an Application to Data-driven Multi-zone Thermal Building Models](https://mlsanity.com/paper/2212.12380/) (2022)
- [Lessons Learned from Data-Driven Building Control Experiments: Contrasting Gaussian Process-based MPC, Bilevel DeePC, and Deep Reinforcement Learning](https://mlsanity.com/paper/2205.15703/) (2022)
- [Computationally Efficient Reinforcement Learning: Targeted Exploration leveraging Simple Rules](https://mlsanity.com/paper/2211.16691/) (2022)

## Abstract (arXiv)

Replacing poorly performing existing controllers with smarter solutions will decrease the energy intensity of the building sector. Recently, controllers based on Deep Reinforcement Learning (DRL) have been shown to be more effective than conventional baselines. However, since the optimal solution is usually unknown, it is still unclear if DRL agents are attaining near-optimal performance in general or if there is still a large gap to bridge. In this paper, we investigate the performance of DRL agents compared to the theoretically optimal solution. To that end, we leverage Physically Consistent Neural Networks (PCNNs) as simulation environments, for which optimal control inputs are easy to compute. Furthermore, PCNNs solely rely on data to be trained, avoiding the difficult physics-based modeling phase, while retaining physical consistency. Our results hint that DRL agents not only clearly outperform conventional rule-based controllers, they furthermore attain near-optimal performance.