# Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1709.05077 · HTML: https://mlsanity.com/paper/1709.05077/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2017.

## Built on by (most cited first, 18 of 18)

- [A Survey of Constraint Formulations in Safe Reinforcement Learning](https://mlsanity.com/paper/2402.02025/) (2024)
- [Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems](https://mlsanity.com/paper/2102.07659/) (2021)
- [Hierarchical Reinforcement Learning for Power Network Topology Control](https://mlsanity.com/paper/2311.02129/) (2023)
- [Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control](https://mlsanity.com/paper/2502.08681/) (2025)
- [An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC control](https://mlsanity.com/paper/2401.05737/) (2024)
- [Data Center Cooling System Optimization Using Offline Reinforcement Learning](https://mlsanity.com/paper/2501.15085/) (2025)
- [From Past to Future: Rethinking Eligibility Traces](https://mlsanity.com/paper/2312.12972/) (2023)
- [FLARE: Fingerprinting Deep Reinforcement Learning Agents using Universal Adversarial Masks](https://mlsanity.com/paper/2307.14751/) (2023)
- [Reinforcement Learning for Sustainable Energy: A Survey](https://mlsanity.com/paper/2407.18597/) (2024)
- [A Provable Approach for End-to-End Safe Reinforcement Learning](https://mlsanity.com/paper/2505.21852/) (2025)
- [DCoPilot: Generative AI-Empowered Policy Adaptation for Dynamic Data Center Operations](https://mlsanity.com/paper/2602.02137/) (2026)
- [Monotonic Neural Network: combining Deep Learning with Domain Knowledge for Chiller Plants Energy Optimization](https://mlsanity.com/paper/2106.06143/) (2021)
- [Development and Validation of an AI-Driven Model for the La Rance Tidal Barrage: A Generalisable Case Study](https://mlsanity.com/paper/2202.05347/) (2022)
- [Experimental evaluation of offline reinforcement learning for HVAC control in buildings](https://mlsanity.com/paper/2408.07986/) (2024)
- [Fusion Intelligence for Digital Twinning AI Data Centers: A Synergistic GenAI-PhyAI Approach](https://mlsanity.com/paper/2505.19409/) (2025)
- [Comparative Field Deployment of Reinforcement Learning and Model Predictive Control for Residential HVAC](https://mlsanity.com/paper/2510.01475/) (2025)
- [Dual-Loop Control in DCVerse: Advancing Reliable Deployment of AI in Data Centers via Digital Twins](https://mlsanity.com/paper/2604.07559/) (2026)
- [Artificial Intelligence for Energy Optimization in Data Centers](https://mlsanity.com/paper/2609.03716/) (2026)

## Abstract (arXiv)

Cooling system plays a critical role in a modern data center (DC). Developing an optimal control policy for DC cooling system is a challenging task. The prevailing approaches often rely on approximating system models that are built upon the knowledge of mechanical cooling, electrical and thermal management, which is difficult to design and may lead to sub-optimal or unstable performances. In this paper, we propose utilizing the large amount of monitoring data in DC to optimize the control policy. To do so, we cast the cooling control policy design into an energy cost minimization problem with temperature constraints, and tap it into the emerging deep reinforcement learning (DRL) framework. Specifically, we propose an end-to-end cooling control algorithm (CCA) that is based on the actor-critic framework and an off-policy offline version of the deep deterministic policy gradient (DDPG) algorithm. In the proposed CCA, an evaluation network is trained to predict an energy cost counter penalized by the cooling status of the DC room, and a policy network is trained to predict optimized control settings when gave the current load and weather information. The proposed algorithm is evaluated on the EnergyPlus simulation platform and on a real data trace collected from the National Super Computing Centre (NSCC) of Singapore. Our results show that the proposed CCA can achieve about 11% cooling cost saving on the simulation platform compared with a manually configured baseline control algorithm. In the trace-based study, we propose a de-underestimation validation mechanism as we cannot directly test the algorithm on a real DC. Even though with DUE the results are conservative, we can still achieve about 15% cooling energy saving on the NSCC data trace if we set the inlet temperature threshold at 26.6 degree Celsius.