mlsanityData to 7 Oct 2026

Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification

arXiv 2502.02133 ↗First posted 4 Feb 2025eess.SY cs.AI cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

The fields of MPC and RL consider two successful control techniques for Markov decision processes. Both approaches are derived from similar fundamental principles, and both are widely used in practical applications, including robotics, process control, energy systems, and autonomous driving. Despite their similarities, MPC and RL follow distinct paradigms that emerged from diverse communities and different requirements. Various technical discrepancies, particularly the role of an environment model as part of the algorithm, lead to methodologies with nearly complementary advantages. Due to their orthogonal benefits, research interest in combination methods has recently increased significantly, leading to a large and growing set of complex ideas leveraging MPC and RL. This work illuminates the differences, similarities, and fundamentals that allow for different combination algorithms and categorizes existing work accordingly. Particularly, we focus on the versatile actor-critic RL approach as a basis for our categorization and examine how the online optimization approach of MPC can be used to improve the overall closed-loop performance of a policy.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 138 of its 331 references that are in the map. First posted on arXiv 4 Feb 2025.

Builds on

138 of 331 references.

References · 138

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  9. 2016Generative Adversarial Imitation Learningcited by 1,576
  10. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  11. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  12. 2017Constrained Policy Optimizationcited by 928
38 more
  1. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  2. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  3. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  4. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  5. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  6. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  7. 2018Maximum a Posteriori Policy Optimisationcited by 318
  8. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  9. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  10. 2018Differentiable MPC for End-to-end Planning and Controlcited by 178
  11. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  12. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  13. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 129
  14. 2018Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Controlcited by 104
  15. 2017Cautious Model Predictive Control using Gaussian Process Regressioncited by 91
  16. 2018A predictive safety filter for learning-based control of constrained nonlinear dynamical systemscited by 91
  17. 2019Exploring Model-based Planning with Policy Networkscited by 90
  18. 2019Safe Reinforcement Learning Using Robust MPCcited by 72
  19. 2023Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learningcited by 72
  20. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 63
  21. 2019acados: a modular open-source framework for fast embedded optimal controlcited by 61
  22. 2019Data Efficient Reinforcement Learning for Legged Robotscited by 60
  23. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  24. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  25. 2018Efficient representation and approximation of model predictive control laws via deep learningcited by 55
  26. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 55
  27. 2018Learning an Approximate Model Predictive Controller with Guaranteescited by 54
  28. 2019Making Deep Q-learning methods robust to time discretizationcited by 51
  29. 2019Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamicscited by 45
  30. 2020Rapidly Adaptable Legged Robots via Evolutionary Meta-Learningcited by 42
  31. 2019Low Level Control of a Quadrotor with Deep Model-Based Reinforcement Learningcited by 40
  32. 2021State Augmented Constrained Reinforcement Learning: Overcoming the Limitations of Learning with Rewardscited by 37
  33. 2023DTC: Deep Tracking Controlcited by 37
  34. 2021Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiationcited by 35
  35. 2018Data-Enabled Predictive Control: In the Shallows of the DeePCcited by 33
  36. 2020Model-Predictive Control via Cross-Entropy and Gradient-Based Optimizationcited by 32
  37. 2022Mirror Learning: A Unifying Framework of Policy Optimisationcited by 31
  38. 2019Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Controlcited by 29

50 most cited of 138 All 138 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 10.

19 papers.

Sort:

By area:

  1. 2025Differentiable Nonlinear Model Predictive Controlcited by 5
  2. 2025MPCritic: A plug-and-play MPC architecture for reinforcement learningcited by 3
  3. 2025Bi-Level Reinforcement Learning Pathway for Sim-to-Real Optimalitycited by 1
  4. 2025Mini Amusement Parks (MAPs): A Testbed for Modelling Business Decisionscited by 1
  5. 2026Sharing the Control Authority Between Deep Reinforcement Learning and Model Predictive Control: Application to Multi-Class Transportation Networkscited by 1
  6. 2025Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agentscited by 0
  7. 2025Computationally efficient Gauss-Newton reinforcement learning for model predictive controlcited by 0
  8. 2025Hierarchical Reinforcement Learning with Low-Level MPC for Multi-Agent Controlcited by 0
  9. 2025Direct transfer of optimized controllers to similar systems using dimensionless MPCcited by 0
  10. 2025CORL: Reinforcement Learning of MILP Policies Solved via Branch and Boundcited by 0
  11. 2026Safety-Guaranteed Imitation Learning from Nonlinear Model Predictive Control for Spacecraft Close Proximity Operationscited by 0
  12. 2026Soft MPCritic: Amortized Model Predictive Value Iterationcited by 0
7 more
  1. 2026A Systematic Review and Taxonomy of Reinforcement Learning-Model Predictive Control Integration for Linear Systemscited by 0
  2. 2026Application of Deep Reinforcement Learning to Event-Triggered Control for Networked Artificial Pancreas Systemscited by 0
  3. 2026Solving Markov Decision Processes with Future Information via MPCcited by 0
  4. 2026Guided Riemannian Optimization (GuRO): Bridging Model Predictive Control and Decision Transformerscited by 0
  5. 2026Accelerating Reinforcement Learning via MPC Solver-Gradient Guidance for Weights-varying MPCcited by 0
  6. 2026Driving Context-guided Model Predictive Planning and Control for Autonomous Car Racing at the Limit and Beyondcited by 0
  7. 2026Composite-Gradient Learning for Shared Control Authority Between Deep Reinforcement Learning and Model Predictive Controlcited by 0