mlsanityData to 14 Oct 2026

End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control

arXiv 2308.01674 ↗First posted 3 Aug 2023cs.LG cs.SY eess.SYcited by 6

Not on the map. Why?

Abstract (arXiv)

(Economic) nonlinear model predictive control ((e)NMPC) requires dynamic models that are sufficiently accurate and computationally tractable. Data-driven surrogate models for mechanistic models can reduce the computational burden of (e)NMPC; however, such models are typically trained by system identification for maximum prediction accuracy on simulation samples and perform suboptimally in (e)NMPC. We present a method for end-to-end reinforcement learning of Koopman surrogate models for optimal performance as part of (e)NMPC. We apply our method to two applications derived from an established nonlinear continuous stirred-tank reactor model. The controller performance is compared to that of (e)NMPCs utilizing models trained using system identification, and model-free neural network controllers trained using reinforcement learning. We show that the end-to-end trained models outperform those trained using system identification in (e)NMPC, and that, in contrast to the neural network controllers, the (e)NMPC controllers can react to changes in the control setting without retraining.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 27 of its 67 references that are in the map. First posted on arXiv 3 Aug 2023.

Builds on

27 of 67 references.

References · 27

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2015Trust Region Policy Optimizationcited by 2,580
  7. 2021Convex optimizationcited by 2,515
  8. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  9. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  10. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 803
  11. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  12. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
15 more
  1. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 494
  2. 2019Differentiable Convex Optimization Layerscited by 465
  3. 2018Differentiable MPC for End-to-end Planning and Controlcited by 189
  4. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 186
  5. 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
  6. 2016Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive controlcited by 140
  7. 2014Learning to Executecited by 106
  8. 2022Accelerated Policy Learning with Parallel Differentiable Simulationcited by 104
  9. 2022Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Timecited by 90
  10. 2019Differentiating Through a Cone Programcited by 73
  11. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 68
  12. 2016Generalizing Koopman Theory to allow for inputs and controlcited by 53
  13. 2021Model Predictive Control of a Vehicle using Koopman Operatorcited by 6
  14. 2021Koopman NMPC: Koopman-based Learning and Nonlinear Model Predictive Control of Control-affine Systemscited by 3
  15. 2022Data-driven End-to-end Learning of Pole Placement Control for Nonlinear Dynamics via Koopman Invariant Subspacescited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 5.

6 papers.

  1. 2025Sample-Efficient Reinforcement Learning of Koopman eNMPCcited by 2
  2. 2024Task-optimal data-driven surrogate models for eNMPC via differentiable simulation and optimizationcited by 1
  3. 2025Deep Neural Koopman Operator-based Economic Model Predictive Control of Shipboard Carbon Capture Systemcited by 0
  4. 2025Computationally efficient Gauss-Newton reinforcement learning for model predictive controlcited by 0
  5. 2025Deep Koopman Economic Model Predictive Control of a Pasteurisation Unitcited by 0
  6. 2025End-to-End Reinforcement Learning of Koopman Models for eNMPC of an Air Separation Unitcited by 0