# SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learning

arXiv: https://arxiv.org/abs/2412.08293 · HTML: https://mlsanity.com/paper/2412.08293/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 37 references that are in the map.
- First posted on arXiv 11 Dec 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 8 of 8)

- [Tackling Climate Change with Machine Learning](https://mlsanity.com/paper/1906.05433/) (2019)
- [A Review of Reinforcement Learning for Autonomous Building Energy Management](https://mlsanity.com/paper/1903.05196/) (2019)
- [CityLearn v2: Energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communities](https://mlsanity.com/paper/2405.03848/) (2024)
- [GridLearn: Multiagent Reinforcement Learning for Grid-Aware Building Energy Management](https://mlsanity.com/paper/2110.06396/) (2021)
- [ModelicaGym: Applying Reinforcement Learning to Modelica Models](https://mlsanity.com/paper/1909.08604/) (2019)
- [PyDCM: Custom Data Center Models with Reinforcement Learning for Sustainability](https://mlsanity.com/paper/2310.03906/) (2023)
- [An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control](https://mlsanity.com/paper/2401.05737/) (2024)
- [B2RL: An open-source Dataset for Building Batch Reinforcement Learning](https://mlsanity.com/paper/2209.15626/) (2022)

## Built on by (most cited first, 5 of 5)

- [Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning](https://mlsanity.com/paper/2607.16534/) (2026)
- [MuFlex: A Scalable, Physics-based Platform for Multi-Building Flexibility Analysis and Coordination](https://mlsanity.com/paper/2508.13532/) (2025)
- [Towards Affordable Energy: A Gymnasium Environment for Electric Utility Demand-Response Programs](https://mlsanity.com/paper/2605.12462/) (2026)
- [Safe Deep Reinforcement Learning for Energy-Efficient HVAC Control in Multi-Zone Residential Buildings](https://mlsanity.com/paper/2608.17235/) (2026)
- [One Agent per Zone: Decentralised Active Inference under Partial Observability for Multi-Zone HVAC Control on Edge Hardware](https://mlsanity.com/paper/2610.19790/) (2026)

## Abstract (arXiv)

Simulation has become a crucial tool for Building Energy Optimization (BEO) as it enables the evaluation of different design and control strategies at a low cost. Machine Learning (ML) algorithms can leverage large-scale simulations to learn optimal control from vast amounts of data without supervision, particularly under the Reinforcement Learning (RL) paradigm. Unfortunately, the lack of open and standardized tools has hindered the widespread application of ML and RL to BEO. To address this issue, this paper presents Sinergym, an open-source Python-based virtual testbed for large-scale building simulation, data collection, continuous control, and experiment monitoring. Sinergym provides a consistent interface for training and running controllers, predefined benchmarks, experiment visualization and replication support, and comprehensive documentation in a ready-to-use software library. This paper 1) highlights the main features of Sinergym in comparison to other existing frameworks, 2) describes its basic usage, and 3) demonstrates its applicability for RL-based BEO through several representative examples. By integrating simulation, data, and control, Sinergym supports the development of intelligent, data-driven applications for more efficient and responsive building operations, aligning with the objectives of digital twin technology.