mlsanityData to 14 Oct 2026

Sample-Efficient Reinforcement Learning of Koopman eNMPC

arXiv 2503.18787 ↗First posted 24 Mar 2025cs.LG math.OCcited by 2

On the map: Deep reinforcement learning methods → RL for autonomous driving and traffic

Abstract (arXiv)

Reinforcement learning (RL) can be used to tune data-driven (economic) nonlinear model predictive controllers ((e)NMPCs) for optimal performance in a specific control task by optimizing the dynamic model or parameters in the policy's objective function or constraints, such as state bounds. However, the sample efficiency of RL is crucial, and to improve it, we combine a model-based RL algorithm with our published method that turns Koopman (e)NMPCs into automatically differentiable policies. We apply our approach to an eNMPC case study of a continuous stirred-tank reactor (CSTR) model from the literature. The approach outperforms benchmark methods, i.e., data-driven eNMPCs using models based on system identification without further RL tuning of the resulting policy, and neural network controllers trained with model-based RL, by achieving superior control performance and higher sample efficiency. Furthermore, utilizing partial prior knowledge about the system dynamics via physics-informed learning further increases sample efficiency.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 47 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

25 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Reinforced Model Predictive Control via Trust-Region Quasi-Newton Policy Optimization28 May

Other references · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  6. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  7. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 494
  8. 2019Differentiable Convex Optimization Layerscited by 465
  9. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 219
  10. 2019Benchmarking Model-Based Reinforcement Learningcited by 162
  11. 2016Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive controlcited by 140
  12. 2018Model-Based Reinforcement Learning via Meta-Policy Optimizationcited by 101
12 more
  1. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 68
  2. 2021Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networkscited by 43
  3. 2021Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Controlcited by 23
  4. 2022Physics-Informed Model-Based Reinforcement Learningcited by 18
  5. 2024Trust the Model Where It Trusts Itself -- Model-Based Actor-Critic with Uncertainty-Aware Rollout Adaptioncited by 18
  6. 2025On Rollouts in Model-Based Reinforcement Learningcited by 10
  7. 2021Robust Model-based Reinforcement Learning for Autonomous Greenhouse Controlcited by 6
  8. 2022Modern Machine Learning Tools for Monitoring and Control of Industrial Processes: A Surveycited by 6
  9. 2023End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Controlcited by 6
  10. 2024Physics-Informed Neural Networks for Dynamic Process Operations with Limited Physical Knowledge and Datacited by 6
  11. 2024Reinforcement Learning for Sustainable Energy: A Surveycited by 2
  12. 2024Task-optimal data-driven surrogate models for eNMPC via differentiable simulation and optimizationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Hyperfastrl: Hypernetwork-based reinforcement learning for unified control of parametric chaotic PDEscited by 0
  2. 2026SKooP: Symmetric Koopman Predictions for Faster and More Generalizable Legged Robot Locomotion with Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025End-to-End Reinforcement Learning of Koopman Models for eNMPC of an Air Separation Unit
  2. 2025Leveraging Reinforcement Learning and Koopman Theory for Enhanced Model Predictive Control Performance
  3. 2024Model Predictive Control of Nonlinear Dynamics Using Online Adaptive Koopman Operators ↗
  4. 2025Computationally efficient Gauss-Newton reinforcement learning for model predictive control
  5. 2024Data-driven optimal control of unknown nonlinear dynamical systems using the Koopman operator
  6. 2024PC-Gym: Benchmark Environments For Process Control Problems
  7. 2026Efficient Reinforcement Learning using Linear Koopman Dynamics for Nonlinear Robotic Systems
  8. 2026Interpreting Reinforcement Learning Model Behavior via Koopman with Control