# Surrogate Empowered Sim2Real Transfer of Deep Reinforcement Learning for ORC Superheat Control

arXiv: https://arxiv.org/abs/2308.02765 · HTML: https://mlsanity.com/paper/2308.02765/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 41 references that are in the map.
- First posted on arXiv 5 Aug 2023.

## Builds on (most cited first, 4 of 4)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Reinforcement Learning for Batch Bioprocess Optimization](https://mlsanity.com/paper/1904.07292/) (2019)
- [Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning](https://mlsanity.com/paper/2111.07171/) (2021)

## Built on by (most cited first, 5 of 5)

- [A Sim2Real Approach for Identifying Task-Relevant Properties in Interpretable Machine Learning](https://mlsanity.com/paper/2406.00116/) (2024)
- [Transfer of Reinforcement Learning-Based Controllers from Model- to Hardware-in-the-Loop](https://mlsanity.com/paper/2310.17671/) (2023)
- [Facilitating Reinforcement Learning for Process Control Using Transfer Learning: Overview and Perspectives](https://mlsanity.com/paper/2404.00247/) (2024)
- [Reinforcement Learning-Driven Plant-Wide Refinery Planning Using Model Decomposition](https://mlsanity.com/paper/2504.08642/) (2025)
- [Multi-Mode Process Control Using Multi-Task Inverse Reinforcement Learning](https://mlsanity.com/paper/2505.21026/) (2025)

## Abstract (arXiv)

The Organic Rankine Cycle (ORC) is widely used in industrial waste heat recovery due to its simple structure and easy maintenance. However, in the context of smart manufacturing in the process industry, traditional model-based optimization control methods are unable to adapt to the varying operating conditions of the ORC system or sudden changes in operating modes. Deep reinforcement learning (DRL) has significant advantages in situations with uncertainty as it directly achieves control objectives by interacting with the environment without requiring an explicit model of the controlled plant. Nevertheless, direct application of DRL to physical ORC systems presents unacceptable safety risks, and its generalization performance under model-plant mismatch is insufficient to support ORC control requirements. Therefore, this paper proposes a Sim2Real transfer learning-based DRL control method for ORC superheat control, which aims to provide a new simple, feasible, and user-friendly solution for energy system optimization control. Experimental results show that the proposed method greatly improves the training speed of DRL in ORC control problems and solves the generalization performance issue of the agent under multiple operating conditions through Sim2Real transfer.