# Human-Like Autonomous Car-Following Model with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1901.00569 · HTML: https://mlsanity.com/paper/1901.00569/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2019.

## Built on by (most cited first, 30 of 48)

- [Data-driven Traffic Simulation: A Comprehensive Review](https://mlsanity.com/paper/2310.15975/) (2023)
- [FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling](https://mlsanity.com/paper/2306.05381/) (2023)
- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles](https://mlsanity.com/paper/2105.14218/) (2021)
- [TransFollower: Long-Sequence Car-Following Trajectory Prediction through Transformer](https://mlsanity.com/paper/2202.03183/) (2022)
- [EnduRL: Enhancing Safety, Stability, and Efficiency of Mixed Traffic Under Real-World Perturbations Via Reinforcement Learning](https://mlsanity.com/paper/2311.12261/) (2023)
- [A Generative Car-following Model Conditioned On Driving Styles](https://mlsanity.com/paper/2112.05399/) (2021)
- [Car-Following Models: A Multidisciplinary Review](https://mlsanity.com/paper/2304.07143/) (2023)
- [Generating and Evolving Reward Functions for Highway Driving with Large Language Models](https://mlsanity.com/paper/2406.10540/) (2024)
- [GenFollower: Enhancing Car-Following Prediction with Large Language Models](https://mlsanity.com/paper/2407.05611/) (2024)
- [Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision](https://mlsanity.com/paper/2109.08880/) (2021)
- [Modeling driver's evasive behavior during safety-critical lane changes:Two-dimensional time-to-collision and deep reinforcement learning](https://mlsanity.com/paper/2209.15133/) (2022)
- [CARL: Congestion-Aware Reinforcement Learning for Imitation-based Perturbations in Mixed Traffic Control](https://mlsanity.com/paper/2404.00796/) (2024)
- [Real-time system optimal traffic routing under uncertainties -- Can physics models boost reinforcement learning?](https://mlsanity.com/paper/2407.07364/) (2024)
- [EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning](https://mlsanity.com/paper/2308.16008/) (2023)
- [From Words to Wheels: Automated Style-Customized Policy Generation for Autonomous Driving](https://mlsanity.com/paper/2409.11694/) (2024)
- [Formulation and validation of a car-following model based on deep reinforcement learning](https://mlsanity.com/paper/2109.14268/) (2021)
- [Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model](https://mlsanity.com/paper/2203.04749/) (2022)
- [Towards Human-Like RL: Taming Non-Naturalistic Behavior in Deep RL via Adaptive Behavioral Costs in 3D Games](https://mlsanity.com/paper/2309.15484/) (2023)
- [Risk-anticipatory autonomous driving strategies considering vehicles' weights, based on hierarchical deep reinforcement learning](https://mlsanity.com/paper/2401.08661/) (2023)
- [MetaFollower: Adaptable Personalized Autonomous Car Following](https://mlsanity.com/paper/2406.16978/) (2024)
- [EditFollower: Tunable Car Following Models for Customizable Adaptive Cruise Control Systems](https://mlsanity.com/paper/2407.02516/) (2024)
- [Perfect Prediction or Plenty of Proposals? What Matters Most in Planning for Autonomous Driving](https://mlsanity.com/paper/2510.15505/) (2025)
- [Markov Regime-Switching Intelligent Driver Model for Interpretable Car-Following Behavior](https://mlsanity.com/paper/2506.14762/) (2025)
- [Vessel-following model for inland waterways based on deep reinforcement learning](https://mlsanity.com/paper/2207.03257/) (2022)
- [A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications](https://mlsanity.com/paper/2210.14524/) (2022)
- [Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer](https://mlsanity.com/paper/2305.11589/) (2023)
- [Learning Car-Following Behaviors Using Bayesian Matrix Normal Mixture Regression](https://mlsanity.com/paper/2404.16023/) (2024)
- [Communication-Aware Reinforcement Learning for Cooperative Adaptive Cruise Control](https://mlsanity.com/paper/2407.08964/) (2024)
- [When Context Is Not Enough: Modeling Unexplained Variability in Car-Following Behavior](https://mlsanity.com/paper/2507.07012/) (2025)

## Abstract (arXiv)

This study proposes a framework for human-like autonomous car-following planning based on deep reinforcement learning (deep RL). Historical driving data are fed into a simulation environment where an RL agent learns from trial and error interactions based on a reward function that signals how much the agent deviates from the empirical data. Through these interactions, an optimal policy, or car-following model that maps in a human-like way from speed, relative speed between a lead and following vehicle, and inter-vehicle spacing to acceleration of a following vehicle is finally obtained. The model can be continuously updated when more data are fed in. Two thousand car-following periods extracted from the 2015 Shanghai Naturalistic Driving Study were used to train the model and compare its performance with that of traditional and recent data-driven car-following models. As shown by this study results, a deep deterministic policy gradient car-following model that uses disparity between simulated and observed speed as the reward function and considers a reaction delay of 1s, denoted as DDPGvRT, can reproduce human-like car-following behavior with higher accuracy than traditional and recent data-driven car-following models. Specifically, the DDPGvRT model has a spacing validation error of 18% and speed validation error of 5%, which are less than those of other models, including the intelligent driver model, models based on locally weighted regression, and conventional neural network-based models. Moreover, the DDPGvRT demonstrates good capability of generalization to various driving situations and can adapt to different drivers by continuously learning. This study demonstrates that reinforcement learning methodology can offer insight into driver behavior and can contribute to the development of human-like autonomous driving algorithms and traffic-flow models.