# On-line Building Energy Optimization using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1707.05878 · HTML: https://mlsanity.com/paper/1707.05878/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2017.

## Built on by (most cited first, 27 of 27)

- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey](https://mlsanity.com/paper/2110.01411/) (2021)
- [Dynamic Sparse Network for Time Series Classification: Learning What to "see''](https://mlsanity.com/paper/2212.09840/) (2022)
- [Sparse Training Theory for Scalable and Efficient Agents](https://mlsanity.com/paper/2103.01636/) (2021)
- [Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learning](https://mlsanity.com/paper/2106.06061/) (2021)
- [Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: simulations and experiments](https://mlsanity.com/paper/2103.01886/) (2021)
- [Massively Digitized Power Grid: Opportunities and Challenges of Use-inspired AI](https://mlsanity.com/paper/2205.05180/) (2022)
- [Federated Reinforcement Learning for Real-Time Electric Vehicle Charging and Discharging Control](https://mlsanity.com/paper/2210.01452/) (2022)
- [An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control](https://mlsanity.com/paper/2401.05737/) (2024)
- [Deep Learning for Intelligent Demand Response and Smart Grids: A Comprehensive Survey](https://mlsanity.com/paper/2101.08013/) (2021)
- [Development of a Soft Actor Critic Deep Reinforcement Learning Approach for Harnessing Energy Flexibility in a Large Office Building](https://mlsanity.com/paper/2104.12125/) (2021)
- [A survey on the development status and application prospects of knowledge graph in smart grids](https://mlsanity.com/paper/2211.00901/) (2022)
- [Optimal Scheduling in IoT-Driven Smart Isolated Microgrids Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2305.00127/) (2023)
- [Reinforcement Learning for Sustainable Energy: A Survey](https://mlsanity.com/paper/2407.18597/) (2024)
- [Delayformer: spatiotemporal transformation for predicting high-dimensional dynamics](https://mlsanity.com/paper/2506.11528/) (2025)
- [A Time-Series Foundation Model by Universal Delay Embedding](https://mlsanity.com/paper/2509.12080/) (2025)
- [Optimal Adaptive Prediction Intervals for Electricity Load Forecasting in Distribution Systems via Reinforcement Learning](https://mlsanity.com/paper/2205.08698/) (2022)
- [Energy Optimization for HVAC Systems in Multi-VAV Open Offices: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2306.13333/) (2023)
- [Reinforcement Learning-based Approach for Vehicle-to-Building Charging with Heterogeneous Agents and Long Term Rewards](https://mlsanity.com/paper/2502.18526/) (2025)
- [Joint Energy Dispatch and Unit Commitment in Microgrids Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2206.01663/) (2022)
- [Domain-adapted Learning and Imitation: DRL for Power Arbitrage](https://mlsanity.com/paper/2301.08360/) (2023)
- [TreeC: a method to generate interpretable energy management systems using a metaheuristic algorithm](https://mlsanity.com/paper/2304.08310/) (2023)
- [Laxity-Aware Scalable Reinforcement Learning for HVAC Control](https://mlsanity.com/paper/2306.16619/) (2023)
- [MuFlex: A Scalable, Physics-based Platform for Multi-Building Flexibility Analysis and Coordination](https://mlsanity.com/paper/2508.13532/) (2025)
- [BuildingGym: An open-source toolbox for AI-based building energy management using reinforcement learning](https://mlsanity.com/paper/2509.11922/) (2025)
- [An Optimal Battery-Free Approach for Emission Reduction by Storing Solar Surplus in Building Thermal Mass](https://mlsanity.com/paper/2603.28217/) (2026)
- [Supervised Reinforcement Learning for the Coordination of Distributed Energy Resources](https://mlsanity.com/paper/2606.24947/) (2026)

## Abstract (arXiv)

Unprecedented high volumes of data are becoming available with the growth of the advanced metering infrastructure. These are expected to benefit planning and operation of the future power system, and to help the customers transition from a passive to an active role. In this paper, we explore for the first time in the smart grid context the benefits of using Deep Reinforcement Learning, a hybrid type of methods that combines Reinforcement Learning with Deep Learning, to perform on-line optimization of schedules for building energy management systems. The learning procedure was explored using two methods, Deep Q-learning and Deep Policy Gradient, both of them being extended to perform multiple actions simultaneously. The proposed approach was validated on the large-scale Pecan Street Inc. database. This highly-dimensional database includes information about photovoltaic power generation, electric vehicles as well as buildings appliances. Moreover, these on-line energy scheduling strategies could be used to provide real-time feedback to consumers to encourage more efficient use of electricity.