mlsanityData to 14 Oct 2026

A Lightweight Calibrated Simulation Enabling Efficient Offline Learning for Optimal Control of Real Buildings

arXiv 2310.08569 ↗First posted 12 Oct 2023cs.AI cs.CE eess.SPcited by 5

Not on the map. Why?

Abstract (arXiv)

Modern commercial Heating, Ventilation, and Air Conditioning (HVAC) devices form a complex and interconnected thermodynamic system with the building and outside weather conditions, and current setpoint control policies are not fully optimized for minimizing energy use and carbon emission. Given a suitable training environment, a Reinforcement Learning (RL) model is able to improve upon these policies, but training such a model, especially in a way that scales to thousands of buildings, presents many real world challenges. We propose a novel simulation-based approach, where a customized simulator is used to train the agent for each building. Our open-source simulator (available online: https://github.com/google/sbsim) is lightweight and calibrated via telemetry from the building to reach a higher level of fidelity. On a two-story, 68,000 square foot building, with 127 devices, we were able to calibrate our simulator to have just over half a degree of drift from the real world over a six-hour interval. This approach is an important step toward having a real-world RL control system that can be scaled to many buildings, allowing for greater efficiency and resulting in reduced energy consumption and carbon emissions.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 6 of its 31 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

6 of 31 references.

References · 6

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2020A Review of Deep Reinforcement Learning for Smart Building Energy Managementcited by 38
  3. 2020Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildingscited by 26
  4. 2019A Review of Reinforcement Learning for Autonomous Building Energy Managementcited by 21
  5. 2019Building HVAC Scheduling Using Reinforcement Learning via Neural Network Based Model Approximationcited by 16
  6. 2023A Comparison of Classical and Deep Reinforcement Learning Methods for HVAC Controlcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024The Smart Buildings Control Suite: A Diverse Open Source Benchmark to Evaluate and Scale HVAC Control Policies for Sustainabilitycited by 5
  2. 2024Constrained Reinforcement Learning for Safe Heat Pump Controlcited by 1
  3. 2025BEAVER: Building Environments with Assessable Variation for Evaluating Multi-Objective Reinforcement Learningcited by 0
  4. 2026PIRS: Physics-Informed Reward Shaping for SAC-Based Building Energy Managementcited by 0
  5. 2026The Smart Buildings Challenge: Learning Foundation Models for Real-World Optimization at Scalecited by 0