# Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Driving

arXiv: https://arxiv.org/abs/1902.00089 · HTML: https://mlsanity.com/paper/1902.00089/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2019.

## Built on by (most cited first, 30 of 59)

- [HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving](https://mlsanity.com/paper/2401.03160/) (2024)
- [FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling](https://mlsanity.com/paper/2306.05381/) (2023)
- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors](https://mlsanity.com/paper/2408.03200/) (2024)
- [Reliably-stabilizing piecewise-affine neural network controllers](https://mlsanity.com/paper/2111.07183/) (2021)
- [CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models](https://mlsanity.com/paper/2502.15119/) (2025)
- [TransFollower: Long-Sequence Car-Following Trajectory Prediction through Transformer](https://mlsanity.com/paper/2202.03183/) (2022)
- [EnduRL: Enhancing Safety, Stability, and Efficiency of Mixed Traffic Under Real-World Perturbations Via Reinforcement Learning](https://mlsanity.com/paper/2311.12261/) (2023)
- [Generating and Evolving Reward Functions for Highway Driving with Large Language Models](https://mlsanity.com/paper/2406.10540/) (2024)
- [Modeling driver's evasive behavior during safety-critical lane changes:Two-dimensional time-to-collision and deep reinforcement learning](https://mlsanity.com/paper/2209.15133/) (2022)
- [Parameterized Decision-making with Multi-modal Perception for Autonomous Driving](https://mlsanity.com/paper/2312.11935/) (2023)
- [CARL: Congestion-Aware Reinforcement Learning for Imitation-based Perturbations in Mixed Traffic Control](https://mlsanity.com/paper/2404.00796/) (2024)
- [Hybrid Car-Following Strategy based on Deep Deterministic Policy Gradient and Cooperative Adaptive Cruise Control](https://mlsanity.com/paper/2103.03796/) (2021)
- [SAINT-ACC: Safety-Aware Intelligent Adaptive Cruise Control for Autonomous Vehicles Using Deep Reinforcement Learning](https://mlsanity.com/paper/2104.06506/) (2021)
- [EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning](https://mlsanity.com/paper/2308.16008/) (2023)
- [RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policies](https://mlsanity.com/paper/2512.01993/) (2025)
- [Deep Reinforcement Learning Aided Platoon Control Relying on V2X Information](https://mlsanity.com/paper/2203.15781/) (2022)
- [Autonomous Platoon Control with Integrated Deep Reinforcement Learning and Dynamic Programming](https://mlsanity.com/paper/2206.07536/) (2022)
- [KI-PMF: Knowledge Integrated Plausible Motion Forecasting](https://mlsanity.com/paper/2310.12007/) (2023)
- [Deep Reinforcement Learning in Autonomous Car Path Planning and Control: A Survey](https://mlsanity.com/paper/2404.00340/) (2024)
- [From Words to Wheels: Automated Style-Customized Policy Generation for Autonomous Driving](https://mlsanity.com/paper/2409.11694/) (2024)
- [Formulation and validation of a car-following model based on deep reinforcement learning](https://mlsanity.com/paper/2109.14268/) (2021)
- [Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model](https://mlsanity.com/paper/2203.04749/) (2022)
- [Reinforcement Learning with a Terminator](https://mlsanity.com/paper/2205.15376/) (2022)
- [Traffic-Aware Autonomous Driving with Differentiable Traffic Simulation](https://mlsanity.com/paper/2210.03772/) (2022)
- [MetaFollower: Adaptable Personalized Autonomous Car Following](https://mlsanity.com/paper/2406.16978/) (2024)
- [EditFollower: Tunable Car Following Models for Customizable Adaptive Cruise Control Systems](https://mlsanity.com/paper/2407.02516/) (2024)
- [Safety in Traffic Management Systems: A Comprehensive Survey](https://mlsanity.com/paper/2308.06204/) (2023)
- [DeepQTest: Testing Autonomous Driving Systems with Reinforcement Learning and Real-world Weather Data](https://mlsanity.com/paper/2310.05170/) (2023)
- [Reinforcement Learning with Continuous Actions Under Unmeasured Confounding](https://mlsanity.com/paper/2505.00304/) (2025)

## Abstract (arXiv)

A model used for velocity control during car following was proposed based on deep reinforcement learning (RL). To fulfil the multi-objectives of car following, a reward function reflecting driving safety, efficiency, and comfort was constructed. With the reward function, the RL agent learns to control vehicle speed in a fashion that maximizes cumulative rewards, through trials and errors in the simulation environment. A total of 1,341 car-following events extracted from the Next Generation Simulation (NGSIM) dataset were used to train the model. Car-following behavior produced by the model were compared with that observed in the empirical NGSIM data, to demonstrate the model's ability to follow a lead vehicle safely, efficiently, and comfortably. Results show that the model demonstrates the capability of safe, efficient, and comfortable velocity control in that it 1) has small percentages (8%) of dangerous minimum time to collision values (textless 5s) than human drivers in the NGSIM data (35%); 2) can maintain efficient and safe headways in the range of 1s to 2s; and 3) can follow the lead vehicle comfortably with smooth acceleration. The results indicate that reinforcement learning methods could contribute to the development of autonomous driving systems.