# Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control

arXiv: https://arxiv.org/abs/2103.03796 · HTML: https://mlsanity.com/paper/2103.03796/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 24 Feb 2021.

## Builds on (most cited first, 8 of 8)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2001.11231/) (2020)
- [A Survey of Deep Learning Applications to Autonomous Vehicle Control](https://mlsanity.com/paper/1912.10773/) (2019)
- [Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1902.00089/) (2019)
- [Dissipation of stop-and-go waves via control of autonomous vehicles: Field experiments](https://mlsanity.com/paper/1705.01693/) (2017)
- [Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control](https://mlsanity.com/paper/1910.12047/) (2019)
- [Longitudinal Dynamic versus Kinematic Models for Car-Following Control Using Deep Reinforcement Learning](https://mlsanity.com/paper/1905.08314/) (2019)
- [A Review on Cooperative Adaptive Cruise Control (CACC) Systems: Architectures, Controls, and Applications](https://mlsanity.com/paper/1809.02867/) (2018)

## Built on by (most cited first, 6 of 6)

- [Deep Reinforcement Learning Aided Platoon Control Relying on V2X Information](https://mlsanity.com/paper/2203.15781/) (2022)
- [Autonomous Platoon Control with Integrated Deep Reinforcement Learning and Dynamic Programming](https://mlsanity.com/paper/2206.07536/) (2022)
- [Multi-Timescale Control and Communications with Deep Reinforcement Learning -- Part I: Communication-Aware Vehicle Control](https://mlsanity.com/paper/2311.11281/) (2023)
- [AVDDPG: Federated reinforcement learning applied to autonomous platoon control](https://mlsanity.com/paper/2207.03484/) (2022)
- [Effective Reinforcement Learning Control using Conservative Soft Actor-Critic](https://mlsanity.com/paper/2505.03356/) (2025)
- [Edge Delayed Deep Deterministic Policy Gradient: efficient continuous control for edge scenarios](https://mlsanity.com/paper/2412.06390/) (2024)

## Abstract (arXiv)

Deep deterministic policy gradient (DDPG)-based car-following strategy can break through the constraints of the differential equation model due to the ability of exploration on complex environments. However, the car-following performance of DDPG is usually degraded by unreasonable reward function design, insufficient training, and low sampling efficiency. In order to solve this kind of problem, a hybrid car-following strategy based on DDPG and cooperative adaptive cruise control (CACC) is proposed. First, the car-following process is modeled as the Markov decision process to calculate CACC and DDPG simultaneously at each frame. Given a current state, two actions are obtained from CACC and DDPG, respectively. Then, an optimal action, corresponding to the one offering a larger reward, is chosen as the output of the hybrid strategy. Meanwhile, a rule is designed to ensure that the change rate of acceleration is smaller than the desired value. Therefore, the proposed strategy not only guarantees the basic performance of car-following through CACC but also makes full use of the advantages of exploration on complex environments via DDPG. Finally, simulation results show that the car-following performance of the proposed strategy is improved compared with that of DDPG and CACC.