mlsanityData to 7 Oct 2026

Modified DDPG car-following model with a real-world human driving experience with CARLA simulator

arXiv 2112.14602 ↗First posted 29 Dec 2021cs.RO cs.AI cs.LGcited by 9

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

In the autonomous driving field, fusion of human knowledge into Deep Reinforcement Learning (DRL) is often based on the human demonstration recorded in a simulated environment. This limits the generalization and the feasibility of application in real-world traffic. We propose a two-stage DRL method to train a car-following agent, that modifies the policy by leveraging the real-world human driving experience and achieves performance superior to the pure DRL agent. Training a DRL agent is done within CARLA framework with Robot Operating System (ROS). For evaluation, we designed different driving scenarios to compare the proposed two-stage DRL car-following agent with other agents. After extracting the "good" behavior from the human driver, the agent becomes more efficient and reasonable, which makes this autonomous agent more suitable for Human-Robot Interaction (HRI) traffic.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 20 of its 60 references that are in the map. First posted on arXiv 29 Dec 2021.

Builds on

20 of 60 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving19 Mar
  2. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving18 Feb

Other references · 18

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  6. 2016End to End Learning for Self-Driving Carscited by 1,000
  7. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  8. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  9. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  10. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  11. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  12. 2019Causal Confusion in Imitation Learningcited by 286
6 more
  1. 2018How to Start Training: The Effect of Initialization and Architecturecited by 109
  2. 2018A Reinforcement Learning Based Approach for Automated Lane Change Maneuverscited by 44
  3. 2021Offline Reinforcement Learning with Pseudometric Learningcited by 28
  4. 2016End-to-End Deep Reinforcement Learning for Lane Keeping Assistcited by 27
  5. 2018Real-time Dynamic Object Detection for Autonomous Driving using Prior 3D-Mapscited by 17
  6. 2021Formulation and validation of a car-following model based on deep reinforcement learningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 1; 2025: 3; 2026: 1.

9 papers.

  1. 2023FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modelingcited by 21
  2. 2023A Platform-Agnostic Deep Reinforcement Learning Framework for Effective Sim2Real Transfer towards Autonomous Drivingcited by 7
  3. 2024Autonomous Driving Small-Scale Cars: A Survey of Recent Developmentcited by 6
  4. 2022Vessel-following model for inland waterways based on deep reinforcement learningcited by 2
  5. 2023Vision-based DRL Autonomous Driving Agent with Sim2Real Transfercited by 2
  6. 2025Deep reinforcement learning-based longitudinal control strategy for automated vehicles at signalised intersectionscited by 1
  7. 2025Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Drivingcited by 1
  8. 2025Knowledge-data fusion dominated vehicle platoon dynamics modeling and analysis: A physics-encoded deep learning approachcited by 0
  9. 2026A Computationally Efficient Learning of Artificial Intelligence System Reliability Considering Error Propagationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learning
  2. 2022Bilateral Deep Reinforcement Learning Approach for Better-than-human Car Following Model
  3. 2020Deep Reinforcement Learning for Autonomous Driving: A Survey
  4. 2023Review of Deep Reinforcement Learning for Autonomous Driving
  5. 2021Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving
  6. 2024HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving
  7. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
  8. 2022Incorporating Voice Instructions in Model-Based Reinforcement Learning for Self-Driving Cars