# A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings

arXiv: https://arxiv.org/abs/2310.08569 · HTML: https://mlsanity.com/paper/2310.08569/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 31 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 6 of 6)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [A Review of Deep Reinforcement Learning for Smart Building Energy Management](https://mlsanity.com/paper/2008.05074/) (2020)
- [Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings](https://mlsanity.com/paper/2006.14156/) (2020)
- [A Review of Reinforcement Learning for Autonomous Building Energy Management](https://mlsanity.com/paper/1903.05196/) (2019)
- [Building HVAC Scheduling Using Reinforcement Learning via Neural Network Based Model Approximation](https://mlsanity.com/paper/1910.05313/) (2019)
- [A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Control](https://mlsanity.com/paper/2308.05711/) (2023)

## Built on by (most cited first, 5 of 5)

- [The Smart Buildings Control Suite: A Diverse Open Source Benchmark to Evaluate and Scale HVAC Control Policies for Sustainability](https://mlsanity.com/paper/2410.03756/) (2024)
- [Constrained Reinforcement Learning for Safe Heat Pump Control](https://mlsanity.com/paper/2409.19716/) (2024)
- [BEAVER: Building Environments with Assessable Variation for Evaluating Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2507.07769/) (2025)
- [PIRS: Physics-Informed Reward Shaping for SAC-Based Building Energy Management](https://mlsanity.com/paper/2605.28232/) (2026)
- [The Smart Buildings Challenge: Learning Foundation Models for Real-World Optimization at Scale](https://mlsanity.com/paper/2610.18511/) (2026)

## Abstract (arXiv)

Modern commercial Heating, Ventilation, and Air Conditioning (HVAC) devices form a complex and interconnected thermodynamic system with the building and outside weather conditions, and current setpoint control policies are not fully optimized for minimizing energy use and carbon emission. Given a suitable training environment, a Reinforcement Learning (RL) model is able to improve upon these policies, but training such a model, especially in a way that scales to thousands of buildings, presents many real world challenges. We propose a novel simulation-based approach, where a customized simulator is used to train the agent for each building. Our open-source simulator (available online: https://github.com/google/sbsim) is lightweight and calibrated via telemetry from the building to reach a higher level of fidelity. On a two-story, 68,000 square foot building, with 127 devices, we were able to calibrate our simulator to have just over half a degree of drift from the real world over a six-hour interval. This approach is an important step toward having a real-world RL control system that can be scaled to many buildings, allowing for greater efficiency and resulting in reduced energy consumption and carbon emissions.