Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
Reinforcement learning (RL) has achieved strong results in control, yet learned policies remain brittle to changes in dynamics, action spaces, observation spaces, or goals, a critical limitation for real-world deployment. Existing benchmarks offer limited diversity and complexity, making it difficult to rigorously study transfer, multi-task learning, and meta-learning in RL. We introduce Building2Building (B2B), a large-scale suite of realistic Heating, Ventilation, and Air Conditioning (HVAC) control environments built on EnergyPlus, a state-of-the-art building simulator. B2B is fully compatible with the Gymnasium interface and features a parametric building generator, enabling the systematic generation of diverse building configurations with heterogeneous observation and action spaces. Based on this suite, we define benchmark tasks targeting key open challenges in RL, including goal adaptation, dynamics adaptation, action-space shifts, and cross-domain transfer. By providing a large-scale, diverse, and physically grounded testbed with standardized evaluation protocols, B2B enables systematic investigation of generalization and transfer in continuous control. Beyond advancing research on generalization in RL, this new benchmark also carries significant societal implications by enabling improved HVAC control at scale, one of the most energy-intensive systems in buildings.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 35 of its 54 references that are in the map. First posted on arXiv 17 Jul 2026.
Builds on
35 of 54 references.
Closest ancestors · 2
Same direction, newest first
- 2024SINERGYM -- A virtual testbed for building energy optimization with Reinforcement Learning11 Dec
- 2024CityLearn v2: Energy-flexible, resilient, occupant-centric, and carbon-aware management of grid-interactive communities2 May
Other references · 33
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
- 2020Gradient Surgery for Multi-Task Learningcited by 1,256
- 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
- 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
- 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
- 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 483
- 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 389
- 2020dm_control: Software and Tasks for Continuous Controlcited by 375
- 2016Successor Features for Transfer in Reinforcement Learningcited by 340
- 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
21 more
- 2021A Practical Guide to Multi-Objective Reinforcement Learning and Planningcited by 256
- 2017Distral: Robust Multitask Reinforcement Learningcited by 233
- 2019Pareto Multi-Task Learningcited by 224
- 2015Contextual Markov Decision Processescited by 220
- 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 217
- 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
- 2021Embodied Intelligence via Learning and Evolutioncited by 163
- 2018Multi-task Deep Reinforcement Learning with PopArtcited by 141
- 2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
- 2016Learning values across many orders of magnitudecited by 85
- 2020My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Controlcited by 70
- 2020VAEM: a Deep Generative Model for Heterogeneous Mixed Type Datacited by 66
- 2021Cross-Domain Imitation Learning via Optimal Transportcited by 49
- 2023Multi-Task Reinforcement Learning with Mixture of Orthogonal Expertscited by 49
- 2025Meta-World+: An Improved, Standardized, RL Benchmarkcited by 43
- 2022AnyMorph: Learning Transferable Polices By Inferring Agent Morphologycited by 27
- 2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
- 2018Reinforcement Learning Testbed for Power-Consumption Optimizationcited by 18
- 2020One for Many: Transfer Learning for Building HVAC Controlcited by 16
- 2025Learning Massively Multitask World Models for Continuous Controlcited by 12
- 2024The Smart Buildings Control Suite: A Diverse Open Source Benchmark to Evaluate and Scale HVAC Control Policies for Sustainabilitycited by 5
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025BEAVER: Building Environments with Assessable Variation for Evaluating Multi-Objective Reinforcement Learning
- 2025Continual Reinforcement Learning for HVAC Systems Control: Integrating Hypernetworks and Transfer Learning
- 2026Generalizing HVAC Control With Domain Randomized Reinforcement Learning
- 2025Reinforcement Learning (RL) Meets Urban Climate Modeling: Investigating the Efficacy and Impacts of RL-Based HVAC Control
- 2026Hierarchical Control Framework Integrating LLMs with RL for Decarbonized HVAC Operation
- 2025Gym4ReaL: A Suite for Benchmarking Real-World Reinforcement Learning
- 2025BuildingGym: An open-source toolbox for AI-based building energy management using reinforcement learning
- 2026Counter-Dyna: Data-Efficient RL-Based HVAC Control using Counterfactual Building Models