# Formulation and validation of a car-following model based on deep reinforcement learning

arXiv: https://arxiv.org/abs/2109.14268 · HTML: https://mlsanity.com/paper/2109.14268/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 29 Sep 2021.

## Builds on (most cited first, 12 of 12)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Modular Multitask Reinforcement Learning with Policy Sketches](https://mlsanity.com/paper/1611.01796/) (2016)
- [Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1902.00089/) (2019)
- [Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning](https://mlsanity.com/paper/1901.00569/) (2019)
- [A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers](https://mlsanity.com/paper/1804.07871/) (2018)
- [Modular Deep Reinforcement Learning for Continuous Motion Planning with Temporal Logic](https://mlsanity.com/paper/2102.12855/) (2021)
- [Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Control](https://mlsanity.com/paper/1910.12047/) (2019)
- [Navigating Occluded Intersections with Autonomous Vehicles using Deep Reinforcement Learning](https://mlsanity.com/paper/1705.01196/) (2017)
- [Comparing Numerical Integration Schemes for Time-Continuous Car-Following Models](https://mlsanity.com/paper/1403.4881/) (2014)
- [Dampen the Stop-and-Go Traffic with Connected and Automated Vehicles -- A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2005.08245/) (2020)
- [Autonomous Ramp Merge Maneuver Based on Reinforcement Learning with Continuous Action Space](https://mlsanity.com/paper/1803.09203/) (2018)

## Built on by (most cited first, 4 of 4)

- [FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling](https://mlsanity.com/paper/2306.05381/) (2023)
- [Modified DDPG car-following model with a real-world human driving experience with CARLA simulator](https://mlsanity.com/paper/2112.14602/) (2021)
- [Vessel-following model for inland waterways based on deep reinforcement learning](https://mlsanity.com/paper/2207.03257/) (2022)
- [A phase-aware AI car-following model for electric vehicles with adaptive cruise control: Development and validation using real-world data](https://mlsanity.com/paper/2510.21735/) (2025)

## Abstract (arXiv)

We propose and validate a novel car following model based on deep reinforcement learning. Our model is trained to maximize externally given reward functions for the free and car-following regimes rather than reproducing existing follower trajectories. The parameters of these reward functions such as desired speed, time gap, or accelerations resemble that of traditional models such as the Intelligent Driver Model (IDM) and allow for explicitly implementing different driving styles. Moreover, they partially lift the black-box nature of conventional neural network models. The model is trained on leading speed profiles governed by a truncated Ornstein-Uhlenbeck process reflecting a realistic leader's kinematics. This allows for arbitrary driving situations and an infinite supply of training data. For various parameterizations of the reward functions, and for a wide variety of artificial and real leader data, the model turned out to be unconditionally string stable, comfortable, and crash-free. String stability has been tested with a platoon of five followers following an artificial and a real leading trajectory. A cross-comparison with the IDM calibrated to the goodness-of-fit of the relative gaps showed a higher reward compared to the traditional model and a better goodness-of-fit.