mlsanityData to 14 Oct 2026

Controlling Commercial Cooling Systems Using Reinforcement Learning

arXiv 2211.07357 ↗First posted 11 Nov 2022cs.LG cs.AI cs.SYcited by 29

Not on the map. Why?

Abstract (arXiv)

This paper is a technical overview of DeepMind and Google's recent work on reinforcement learning for controlling commercial cooling systems. Building on expertise that began with cooling Google's data centers more efficiently, we recently conducted live experiments on two real-world facilities in partnership with Trane Technologies, a building management system provider. These live experiments had a variety of challenges in areas such as evaluation, learning from offline data, and constraint satisfaction. Our paper describes these challenges in the hope that awareness of them will benefit future applied RL work. We also describe the way we adapted our RL system to deal with these challenges, resulting in energy savings of approximately 9% and 13% respectively at the two live experiment sites.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 13 of its 29 references that are in the map. First posted on arXiv 11 Nov 2022.

Builds on

13 of 29 references.

References · 13

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  5. 2016Deep Exploration via Bootstrapped DQNcited by 527
  6. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  7. 2019Challenges of Real-World Reinforcement Learningcited by 291
  8. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 168
  9. 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 76
  10. 2020A Review of Deep Reinforcement Learning for Smart Building Energy Managementcited by 38
  11. 2021Regularized Behavior Value Estimationcited by 32
  12. 2022Semi-analytical Industrial Cooling System Model for Reinforcement Learningcited by 9
1 more
  1. 2018TD or not TD: Analyzing the Role of Temporal Differencing in Deep Reinforcement Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 9; 2025: 6; 2026: 5.

29 papers.

Sort:

By area:

  1. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  2. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 60
  3. 2023ReLOAD: Reinforcement Learning with Optimistic Ascent-Descent for Last-Iterate Convergence in Constrained MDPscited by 20
  4. 2025MONA: Myopic Optimization with Non-myopic Approval Can Mitigate Multi-step Reward Hackingcited by 19
  5. 2023GVFs in the Real World: Making Predictions Online for Water Treatmentcited by 13
  6. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 12
  7. 2024Position: Lifetime tuning is incompatible with continual reinforcement learningcited by 11
  8. 2024A Review of Safe Reinforcement Learning Methods for Modern Power Systemscited by 11
  9. 2023Towards practical reinforcement learning for tokamak magnetic controlcited by 9
  10. 2024CoDreamer: Communication-Based Decentralised World Modelscited by 5
  11. 2024Stealthy Imitation: Reward-guided Environment-free Policy Stealingcited by 3
  12. 2024State and Action Factorization in Power Gridscited by 3
17 more
  1. 2025Lessons learned from field demonstrations of model predictive control and reinforcement learning for residential and commercial HVAC: A reviewcited by 3
  2. 2026AI Finds A Waycited by 3
  3. 2024Transfer Learning on Transformers for Building Energy Consumption Forecasting -- A Comparative Studycited by 2
  4. 2023Reinforcement Learning Based Power Grid Day-Ahead Planning and AI-Assisted Controlcited by 1
  5. 2023Anytime-Competitive Reinforcement Learning with Policy Priorcited by 1
  6. 2024Black Box Meta-Learning Intrinsic Rewardscited by 1
  7. 2025Nuclear Microreactor Control with Deep Reinforcement Learningcited by 1
  8. 2025Control of Renewable Energy Communities using AI and Real-World Datacited by 1
  9. 2026Action Shapley: A Training Data Selection Metric for World Model in Reinforcement Learningcited by 1
  10. 2026Position: RL Researchers Need to Distinguish Between Solving Simulators and Using Simulators as a Proxycited by 1
  11. 2023Learning Visual Tracking and Reaching with Deep Reinforcement Learning on a UR10e Robotic Armcited by 0
  12. 2024Interactive Design-of-Experiments: Optimizing a Cooling Systemcited by 0
  13. 2024Effective Reward Specification in Deep Reinforcement Learningcited by 0
  14. 2025Comparative Field Deployment of Reinforcement Learning and Model Predictive Control for Residential HVACcited by 0
  15. 2025A Receding Horizon Reinforcement Learning Framework for Campus Chiller Energy Management - A case study from an Australian Universitycited by 0
  16. 2026Dynamics Models for Offline Hyperparameter Selection in Real-World RLcited by 0
  17. 2026Admissable: Training Reinforcement Learning Agents against Adversarial Missingnesscited by 0