# Offline-Online Reinforcement Learning for Energy Pricing in Office Demand Response: Lowering Energy and Data Costs

arXiv: https://arxiv.org/abs/2108.06594 · HTML: https://mlsanity.com/paper/2108.06594/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 44 references that are in the map.
- First posted on arXiv 14 Aug 2021.

## Builds on (most cited first, 6 of 6)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [A Novel Graphical Lasso based approach towards Segmentation Analysis in Energy Game-Theoretic Frameworks](https://mlsanity.com/paper/1910.02217/) (2019)
- [Design, Benchmarking and Explainability Analysis of a Game-Theoretic Framework towards Energy Efficiency in Smart Infrastructure](https://mlsanity.com/paper/1910.07899/) (2019)
- [A Robust Utility Learning Framework via Inverse Optimization](https://mlsanity.com/paper/1704.07933/) (2017)
- [Using Meta Reinforcement Learning to Bridge the Gap between Simulation and Experiment in Energy Demand Response](https://mlsanity.com/paper/2104.14670/) (2021)

## Built on by (most cited first, 3 of 3)

- [Active Reinforcement Learning for Robust Building Control](https://mlsanity.com/paper/2312.10289/) (2023)
- [Machine Learning for Smart and Energy-Efficient Buildings](https://mlsanity.com/paper/2211.14889/) (2022)
- [FedAR+: A Federated Learning Approach to Appliance Recognition with Mislabeled Data in Residential Buildings](https://mlsanity.com/paper/2209.01338/) (2022)

## Abstract (arXiv)

Our team is proposing to run a full-scale energy demand response experiment in an office building. Although this is an exciting endeavor which will provide value to the community, collecting training data for the reinforcement learning agent is costly and will be limited. In this work, we examine how offline training can be leveraged to minimize data costs (accelerate convergence) and program implementation costs. We present two approaches to doing so: pretraining our model to warm start the experiment with simulated tasks, and using a planning model trained to simulate the real world's rewards to the agent. We present results that demonstrate the utility of offline reinforcement learning to efficient price-setting in the energy demand response problem.