# Hierarchical RL-MPC for Demand Response Scheduling

arXiv: https://arxiv.org/abs/2502.13714 · HTML: https://mlsanity.com/paper/2502.13714/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 3 of its 16 references that are in the map.
- First posted on arXiv 19 Feb 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 3 of 3)

- [Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning](https://mlsanity.com/paper/2111.07171/) (2021)
- [Chance Constrained Policy Optimization for Process Control and Optimization](https://mlsanity.com/paper/2008.00030/) (2020)
- [Control-Informed Reinforcement Learning for Chemical Processes](https://mlsanity.com/paper/2408.13566/) (2024)

## Built on by (most cited first, 3 of 3)

- [Survey and Tutorial of Reinforcement Learning Methods in Process Systems Engineering](https://mlsanity.com/paper/2510.24272/) (2025)
- [Addressing Terminal Constraints in Data-Driven Demand Response Scheduling](https://mlsanity.com/paper/2605.14741/) (2026)
- [Atoms to Processes: The Role of Artificial Intelligence and Machine Learning in Chemical Engineering](https://mlsanity.com/paper/2610.02014/) (2026)

## Abstract (arXiv)

This paper presents a hierarchical framework for demand response optimization in air separation units (ASUs) that combines reinforcement learning (RL) with linear model predictive control (LMPC). We investigate two control architectures: a direct RL approach and a control-informed methodology where an RL agent provides setpoints to a lower-level LMPC. The proposed RL-LMPC framework demonstrates improved sample efficiency during training and better constraint satisfaction compared to direct RL control. Using an industrial ASU case study, we show that our approach successfully manages operational constraints while optimizing electricity costs under time-varying pricing. Results indicate that the RL-LMPC architecture achieves comparable economic performance to direct RL while providing better robustness and requiring fewer training samples to converge. The framework offers a practical solution for implementing flexible operation strategies in process industries, bridging the gap between data-driven methods and traditional control approaches.