# Semi-analytical Industrial Cooling System Model for Reinforcement Learning

arXiv: https://arxiv.org/abs/2207.13131 · HTML: https://mlsanity.com/paper/2207.13131/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 26 Jul 2022.

## Builds on (most cited first, 14 of 14)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Maximum a Posteriori Policy Optimisation](https://mlsanity.com/paper/1806.06920/) (2018)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [Distributed Distributional Deterministic Policy Gradients](https://mlsanity.com/paper/1804.08617/) (2018)
- [Acme: A Research Framework for Distributed Reinforcement Learning](https://mlsanity.com/paper/2006.00979/) (2020)
- [Optimal Control Via Neural Networks: A Convex Approach](https://mlsanity.com/paper/1805.11835/) (2018)
- [DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning](https://mlsanity.com/paper/2102.11492/) (2021)
- [Reinforcement Learning Testbed for Power-Consumption Optimization](https://mlsanity.com/paper/1808.10427/) (2018)
- [Energy-Efficient Thermal Comfort Control in Smart Buildings via Deep Reinforcement Learning](https://mlsanity.com/paper/1901.04693/) (2019)
- [Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control](https://mlsanity.com/paper/2110.13450/) (2021)

## Built on by (most cited first, 9 of 9)

- [Controlling Commercial Cooling Systems Using Reinforcement Learning](https://mlsanity.com/paper/2211.07357/) (2022)
- [Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design](https://mlsanity.com/paper/2301.13734/) (2023)
- [Efficient Multi-Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/2408.08706/) (2024)
- [Doubly Optimal Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/2410.02226/) (2024)
- [Efficient Policy Evaluation with Safety Constraint for Reinforcement Learning](https://mlsanity.com/paper/2410.05655/) (2024)
- [Data Center Cooling System Optimization Using Offline Reinforcement Learning](https://mlsanity.com/paper/2501.15085/) (2025)
- [Optimizing Industrial HVAC Systems with Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2209.08112/) (2022)
- [Anytime-Competitive Reinforcement Learning with Policy Prior](https://mlsanity.com/paper/2311.01568/) (2023)
- [Interactive Design-of-Experiments: Optimizing a Cooling System](https://mlsanity.com/paper/2408.12607/) (2024)

## Abstract (arXiv)

We present a hybrid industrial cooling system model that embeds analytical solutions within a multi-physics simulation. This model is designed for reinforcement learning (RL) applications and balances simplicity with simulation fidelity and interpretability. The model's fidelity is evaluated against real world data from a large scale cooling system. This is followed by a case study illustrating how the model can be used for RL research. For this, we develop an industrial task suite that allows specifying different problem settings and levels of complexity, and use it to evaluate the performance of different RL algorithms.