mlsanityData to 7 Oct 2026

Prioritized Experience-based Reinforcement Learning with Human Guidance for Autonomous Driving

arXiv 2109.12516 ↗First posted 26 Sep 2021cs.LG cs.ROcited by 21

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Reinforcement learning (RL) requires skillful definition and remarkable computational efforts to solve optimization and control problems, which could impair its prospect. Introducing human guidance into reinforcement learning is a promising way to improve learning performance. In this paper, a comprehensive human guidance-based reinforcement learning framework is established. A novel prioritized experience replay mechanism that adapts to human guidance in the reinforcement learning process is proposed to boost the efficiency and performance of the reinforcement learning algorithm. To relieve the heavy workload on human participants, a behavior model is established based on an incremental online learning method to mimic human actions. We design two challenging autonomous driving tasks for evaluating the proposed algorithm. Experiments are conducted to access the training and testing performance and learning mechanism of the proposed algorithm. Comparative results against the state-of-the-art methods suggest the advantages of our algorithm in terms of learning efficiency, performance, and robustness.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 16 of its 36 references that are in the map. First posted on arXiv 26 Sep 2021.

Builds on

16 of 36 references.

References · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2015Prioritized Experience Replaycited by 1,070
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  7. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  8. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  9. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  10. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  11. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  12. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 102
4 more
  1. 2019Making Efficient Use of Demonstrations to Solve Hard Exploration Problemscited by 45
  2. 2020Guided Exploration with Proximal Policy Optimization using a Single Demonstrationcited by 19
  3. 2018Intervention Aided Reinforcement Learning for Safe and Practical Policy Optimization in Navigationcited by 12
  4. 2020Reinforcement Learning Control of Robotic Knee with Human in the Loop by Flexible Policy Iterationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 4; 2024: 4; 2025: 8; 2026: 2.

21 papers.

Sort:

By area:

  1. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  2. 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Drivingcited by 34
  3. 2024Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Drivingcited by 8
  4. 2025Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Drivingcited by 7
  5. 2023Learning Interaction-aware Motion Prediction Model for Decision-making in Autonomous Drivingcited by 6
  6. 2024Pre-trained Transformer-Enabled Strategies with Human-Guided Fine-Tuning for End-to-end Navigation of Autonomous Vehiclescited by 6
  7. 2024Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunitiescited by 6
  8. 2022Safe Decision-making for Lane-change of Autonomous Vehicles via Human Demonstration-aided Reinforcement Learningcited by 5
  9. 2025LearningFlow: Automated Policy Learning Workflow for Urban Driving with Large Language Modelscited by 4
  10. 2024Enhancing End-to-End Autonomous Driving Systems Through Synchronized Human Behavior Datacited by 3
  11. 2025Multi-Objective Reinforcement Learning for Adaptable Personalized Autonomous Drivingcited by 3
  12. 2022Sampling Efficient Deep Reinforcement Learning through Preference-Guided Stochastic Explorationcited by 2
9 more
  1. 2023Magnetic Field-Based Reward Shaping for Goal-Conditioned Reinforcement Learningcited by 2
  2. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspectivecited by 2
  3. 2025AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systemscited by 2
  4. 2026Neuro-Cognitive Reward Modeling for Human-Centered Autonomous Vehicle Controlcited by 2
  5. 2023Multi-Agent Probabilistic Ensembles with Trajectory Sampling for Connected Autonomous Vehiclescited by 1
  6. 2025A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approachcited by 1
  7. 2025Multi-agent Uncertainty-Aware Pessimistic Model-Based Reinforcement Learning for Connected Autonomous Vehiclescited by 0
  8. 2025Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Drivingcited by 0
  9. 2026ClaHF: A Human Feedback-inspired Reinforcement Learning Framework for Improving Classification Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Autonomous Algorithm for Training Autonomous Vehicles with Minimal Human Intervention
  2. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
  3. 2025From Learning to Mastery: Achieving Safe and Efficient Real-World Autonomous Driving with Human-In-The-Loop Reinforcement Learning
  4. 2019Leveraging Human Guidance for Deep Reinforcement Learning Tasks
  5. 2021Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving
  6. 2022Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skills
  7. 2023Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approach
  8. 2025Towards Human-Centric Autonomous Driving: A Fast-Slow Architecture Integrating Large Language Model Guidance with Reinforcement Learning