mlsanityData to 14 Oct 2026

HOFLON: Hybrid Offline Learning and Online Optimization for Process Start-Up and Grade-Transition Control

arXiv 2510.03830 ↗First posted 4 Oct 2025cs.LG cs.SY eess.SYcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

also in 1 more
Abstract (arXiv)

Start-ups and product grade-changes are critical steps in continuous-process plant operation, because any misstep immediately affects product quality and drives operational losses. These transitions have long relied on manual operation by a handful of expert operators, but the progressive retirement of that workforce is leaving plant owners without the tacit know-how needed to execute them consistently. In the absence of a process model, offline reinforcement learning (RL) promises to capture and even surpass human expertise by mining historical start-up and grade-change logs, yet standard offline RL struggles with distribution shift and value-overestimation whenever a learned policy ventures outside the data envelope. We introduce HOFLON (Hybrid Offline Learning + Online Optimization) to overcome those limitations. Offline, HOFLON learns (i) a latent data manifold that represents the feasible region spanned by past transitions and (ii) a long-horizon Q-critic that predicts the cumulative reward from state-action pairs. Online, it solves a one-step optimization problem that maximizes the Q-critic while penalizing deviations from the learned manifold and excessive rates of change in the manipulated variables. We test HOFLON on two industrial case studies: a polymerization reactor start-up and a paper-machine grade-change problem, and benchmark it against Implicit Q-Learning (IQL), a leading offline-RL algorithm. In both plants HOFLON not only surpasses IQL but also delivers, on average, better cumulative rewards than the best start-up or grade-change observed in the historical data, demonstrating its potential to automate transition operations beyond current expert capability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 20 of its 49 references that are in the map. First posted on arXiv 4 Oct 2025.

Builds on

20 of 49 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Taming OOD Actions for Offline Reinforcement Learning: An Advantage-Based Approach8 May
  2. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems2 Mar
  3. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  4. 2020PLAS: Latent Action Space for Offline Reinforcement Learning14 Nov
  5. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  7. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 13

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  4. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  5. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  6. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  7. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  8. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  9. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  10. 2016Input Convex Neural Networkscited by 414
  11. 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 157
  12. 2024Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learningcited by 42
1 more
  1. 2025Prior-Guided Diffusion Planning for Offline Reinforcement Learningcited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PORL: Pretrained Offline Reinforcement Learning for the Job Shop Scheduling Problem
  2. 2025Generalizing Beyond Suboptimality: Offline Reinforcement Learning Learns Effective Scheduling through Random Solutions
  3. 2025Safe Deployment of Offline Reinforcement Learning via Input Convex Action Correction
  4. 2026When Does Trajectory-Level Supervision Permit Efficient Offline Reinforcement Learning?
  5. 2024Offline Reinforcement Learning for Learning to Dispatch for Job Shop Scheduling
  6. 2025LineFlow: A Framework to Learn Active Control of Production Lines
  7. 2026JumpStart Your Policy Learning with Lessons from 160,000 Training Runs
  8. 2025Beyond Non-Expert Demonstrations: Outcome-Driven Action Constraint for Offline Reinforcement Learning