# Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control

arXiv: https://arxiv.org/abs/1910.12047 · HTML: https://mlsanity.com/paper/1910.12047/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2019.

## Built on by (most cited first, 22 of 22)

- [Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification](https://mlsanity.com/paper/2502.02133/) (2025)
- [Event-Triggered Model Predictive Control with Deep Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2208.10302/) (2022)
- [Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control](https://mlsanity.com/paper/2103.03796/) (2021)
- [Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System](https://mlsanity.com/paper/2408.15633/) (2024)
- [Deep Reinforcement Learning Aided Platoon Control Relying on V2X Information](https://mlsanity.com/paper/2203.15781/) (2022)
- [Autonomous Platoon Control with Integrated Deep Reinforcement Learning and Dynamic Programming](https://mlsanity.com/paper/2206.07536/) (2022)
- [Formulation and validation of a car-following model based on deep reinforcement learning](https://mlsanity.com/paper/2109.14268/) (2021)
- [Risk-anticipatory autonomous driving strategies considering vehicles' weights, based on hierarchical deep reinforcement learning](https://mlsanity.com/paper/2401.08661/) (2023)
- [A Deep Reinforcement Learning Based Motion Cueing Algorithm for Vehicle Driving Simulation](https://mlsanity.com/paper/2304.07600/) (2023)
- [Tactical Decision Making for Autonomous Trucks by Deep Reinforcement Learning with Total Cost of Operation Based Reward](https://mlsanity.com/paper/2403.06524/) (2024)
- [Benchmarking Reinforcement Learning Methods for Dexterous Robotic Manipulation with a Three-Fingered Gripper](https://mlsanity.com/paper/2408.14747/) (2024)
- [Vessel-following model for inland waterways based on deep reinforcement learning](https://mlsanity.com/paper/2207.03257/) (2022)
- [Multi-Timescale Control and Communications with Deep Reinforcement Learning -- Part I: Communication-Aware Vehicle Control](https://mlsanity.com/paper/2311.11281/) (2023)
- [Communication-Aware Reinforcement Learning for Cooperative Adaptive Cruise Control](https://mlsanity.com/paper/2407.08964/) (2024)
- [AVDDPG: Federated reinforcement learning applied to autonomous platoon control](https://mlsanity.com/paper/2207.03484/) (2022)
- [Neural Network Based Model Predictive Control for an Autonomous Vehicle](https://mlsanity.com/paper/2107.14573/) (2021)
- [Steady-State Error Compensation for Reinforcement Learning with Quadratic Rewards](https://mlsanity.com/paper/2402.09075/) (2024)
- [Should we use model-free or model-based control? A case study of battery management systems](https://mlsanity.com/paper/2407.15313/) (2024)
- [MORTAR: A Model-based Runtime Action Repair Framework for AI-enabled Cyber-Physical Systems](https://mlsanity.com/paper/2408.03892/) (2024)
- [A nonlinear real time capable motion cueing algorithm based on deep reinforcement learning](https://mlsanity.com/paper/2503.10419/) (2025)
- [Tackling Snow-Induced Challenges: Safe Autonomous Lane-Keeping with Robust Reinforcement Learning](https://mlsanity.com/paper/2512.12987/) (2025)
- [To Learn or Not to Learn: A Litmus Test for Using Reinforcement Learning in Control](https://mlsanity.com/paper/2604.11463/) (2026)

## Abstract (arXiv)

This study compares Deep Reinforcement Learning (DRL) and Model Predictive Control (MPC) for Adaptive Cruise Control (ACC) design in car-following scenarios. A first-order system is used as the Control-Oriented Model (COM) to approximate the acceleration command dynamics of a vehicle. Based on the equations of the control system and the multi-objective cost function, we train a DRL policy using Deep Deterministic Policy Gradient (DDPG) and solve the MPC problem via Interior-Point Optimization (IPO). Simulation results for the episode costs show that, when there are no modeling errors and the testing inputs are within the training data range, the DRL solution is equivalent to MPC with a sufficiently long prediction horizon. Particularly, the DRL episode cost is only 5.8% higher than the benchmark solution provided by optimizing the entire episode via IPO. The DRL control performance degrades when the testing inputs are outside the training data range, indicating inadequate generalization. When there are modeling errors due to control delays, disturbances, and/or testing with a High-Fidelity Model (HFM) of the vehicle, the DRL-trained policy performs better with large modeling errors while having similar performance as MPC when the modeling errors are small.