mlsanityData to 14 Oct 2026

Reinforcement Learning with Model Predictive Control for Highway Ramp Metering

arXiv 2311.08820 ↗First posted 15 Nov 2023eess.SY cs.AI cs.SYcited by 4

Not on the map. Why?

Abstract (arXiv)

In the backdrop of an increasingly pressing need for effective urban and highway transportation systems, this work explores the synergy between model-based and learning-based strategies to enhance traffic flow management by use of an innovative approach to the problem of ramp metering control that embeds Reinforcement Learning (RL) techniques within the Model Predictive Control (MPC) framework. The control problem is formulated as an RL task by crafting a suitable stage cost function that is representative of the traffic conditions, variability in the control action, and violations of the constraint on the maximum number of vehicles in queue. An MPC-based RL approach, which leverages the MPC optimal problem as a function approximation for the RL algorithm, is proposed to learn to efficiently control an on-ramp and satisfy its constraints despite uncertainties in the system model and variable demands. Simulations are performed on a benchmark small-scale highway network to compare the proposed methodology against other state-of-the-art control approaches. Results show that, starting from an MPC controller that has an imprecise model and is poorly tuned, the proposed methodology is able to effectively learn to improve the control policy such that congestion in the network is reduced and constraints are satisfied, yielding an improved performance that is superior to the other controllers.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 13 of its 80 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

13 of 80 references.

References · 13

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 405
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  4. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  5. 2017Cautious Model Predictive Control using Gaussian Process Regressioncited by 91
  6. 2019Safe Reinforcement Learning Using Robust MPCcited by 85
  7. 2019Data-driven Economic NMPC using Reinforcement Learningcited by 68
  8. 2019Performance-oriented model learning for data-driven MPC designcited by 22
  9. 2019Probabilistic model predictive safety certification for learning-based controlcited by 22
  10. 2017Expert Level control of Ramp Metering based on Multi-task Deep Reinforcement Learningcited by 16
  11. 2020Learning for MPC with Stability & Safety Guaranteescited by 14
  12. 2020Model-Reference Reinforcement Learning for Collision-Free Tracking Control of Autonomous Surface Vehiclescited by 11
1 more
  1. 2021Approximate Robust NMPC using Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Safe model-based Reinforcement Learning via Model Predictive Control and Control Barrier Functionscited by 1
  2. 2025Second-Order MPC-Based Distributed Q-Learningcited by 0
  3. 2025Adaptive Tuning of Parameterized Traffic Controllers via Multi-Agent Reinforcement Learningcited by 0
  4. 2026LLM-Based Hierarchical Coordinated Control with Continuation-Aware Policy Learningcited by 0