mlsanityData to 7 Oct 2026

MPC-Inspired Reinforcement Learning for Verifiable Model-Free Control

arXiv 2312.05332 ↗First posted 8 Dec 2023eess.SY cs.LG cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce a new class of parameterized controllers, drawing inspiration from Model Predictive Control (MPC). The controller resembles a Quadratic Programming (QP) solver of a linear MPC problem, with the parameters of the controller being trained via Deep Reinforcement Learning (DRL) rather than derived from system models. This approach addresses the limitations of common controllers with Multi-Layer Perceptron (MLP) or other general neural network architecture used in DRL, in terms of verifiability and performance guarantees, and the learned controllers possess verifiable properties like persistent feasibility and asymptotic stability akin to MPC. On the other hand, numerical examples illustrate that the proposed controller empirically matches MPC and MLP controllers in terms of control performance and has superior robustness against modeling uncertainty and noises. Furthermore, the proposed controller is significantly more computationally efficient compared to MPC and requires fewer parameters to learn than MLP controllers. Real-world experiments on vehicle drift maneuvering task demonstrate the potential of these controllers for robotics and other demanding control tasks.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 39 of its 76 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

39 of 76 references.

References · 39

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2018World Modelscited by 1,444
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  6. 2017Constrained Policy Optimizationcited by 928
  7. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  8. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 482
  9. 2019Differentiable Convex Optimization Layerscited by 458
  10. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 411
  11. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 395
  12. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 361
27 more
  1. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  2. 2018Differentiable MPC for End-to-end Planning and Controlcited by 178
  3. 2019Active Domain Randomizationcited by 161
  4. 2017OSQP: An Operator Splitting Solver for Quadratic Programscited by 156
  5. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  6. 2020Neural Lyapunov Controlcited by 142
  7. 2022Rapid Locomotion via Reinforcement Learningcited by 130
  8. 2021Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPscited by 115
  9. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 95
  10. 2017Cautious Model Predictive Control using Gaussian Process Regressioncited by 91
  11. 2021Lyapunov-stable neural-network controlcited by 66
  12. 2019Deep Drone Racing: From Simulation to Reality with Domain Randomizationcited by 63
  13. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 55
  14. 2022Reachability Constrained Reinforcement Learningcited by 52
  15. 2022Neural Lyapunov Control of Unknown Nonlinear Systems with Stability Guaranteescited by 39
  16. 2016Learning Model Predictive Control for iterative tasks. A Data-Driven Control Frameworkcited by 38
  17. 2018Feedback Control For Cassie With Deep Reinforcement Learningcited by 36
  18. 2019Deep Lagrangian Networks for end-to-end learning of energy-based control for under-actuated systemscited by 32
  19. 2022Neural Koopman Lyapunov Controlcited by 11
  20. 2022Stability Verification of Neural Network Controllers using Mixed-Integer Programmingcited by 10
  21. 2018Structured Control Nets for Deep Reinforcement Learningcited by 8
  22. 2019Efficient Calibration of Embedded MPCcited by 6
  23. 2021Two-timescale Mechanism-and-Data-Driven Control for Aggressive Driving of Autonomous Carscited by 2
  24. 2021A Hierarchical Control Framework for Drift Maneuvering of Autonomous Vehiclescited by 2
  25. 2023Linear Model Predictive Control under Continuous Path Constraints via Parallelized Primal-Dual Hybrid Gradient Algorithmcited by 2
  26. 2023Consecutive Inertia Drift of Autonomous RC Car via Primitive-based Planning and Data-driven Controlcited by 2
  27. 2019Quickly Finding the Best Linear Model in High Dimensionscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026RAY-TOLD: Ray-Based Latent Dynamics for Dense Dynamic Obstacle Avoidance with TDMPCcited by 0
  2. 2026Hierarchical Decision Making with Structured Policies: A Principled Design via Inverse Optimizationcited by 0