mlsanityData to 7 Oct 2026

HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving

arXiv 2401.03160 ↗First posted 6 Jan 2024cs.LG cs.AI cs.ROcited by 31

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Despite significant progress in autonomous vehicles (AVs), the development of driving policies that ensure both the safety of AVs and traffic flow efficiency has not yet been fully explored. In this paper, we propose an enhanced human-in-the-loop reinforcement learning method, termed the Human as AI mentor-based deep reinforcement learning (HAIM-DRL) framework, which facilitates safe and efficient autonomous driving in mixed traffic platoon. Drawing inspiration from the human learning process, we first introduce an innovative learning paradigm that effectively injects human intelligence into AI, termed Human as AI mentor (HAIM). In this paradigm, the human expert serves as a mentor to the AI agent. While allowing the agent to sufficiently explore uncertain environments, the human expert can take control in dangerous situations and demonstrate correct actions to avoid potential accidents. On the other hand, the agent could be guided to minimize traffic flow disturbance, thereby optimizing traffic flow efficiency. In detail, HAIM-DRL leverages data collected from free exploration and partial human demonstrations as its two training sources. Remarkably, we circumvent the intricate process of manually designing reward functions; instead, we directly derive proxy state-action values from partial human demonstrations to guide the agents' policy learning. Additionally, we employ a minimal intervention technique to reduce the human mentor's cognitive load. Comparative results show that HAIM-DRL outperforms traditional methods in driving safety, sampling efficiency, mitigation of traffic flow disturbance, and generalizability to unseen traffic scenarios. The code and demo videos for this paper can be accessed at: https://zilin-huang.github.io/HAIM-DRL-website/

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 33 of its 84 references that are in the map. First posted on arXiv 6 Jan 2024.

Builds on

33 of 84 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Safe Decision-making for Lane-change of Autonomous Vehicles via Human Demonstration-aided Reinforcement Learning1 Jul
  2. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization17 Feb
  3. 2021Safe Driving via Expert Guided Policy Optimization13 Oct
  4. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep
  5. 2021Reward (Mis)design for Autonomous Driving28 Apr
  6. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learning6 Jan

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  8. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  9. 2016Generative Adversarial Imitation Learningcited by 1,576
  10. 2017Constrained Policy Optimizationcited by 928
  11. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  12. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
15 more
  1. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  2. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  3. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  4. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 245
  5. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 103
  6. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 103
  7. 2020Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehiclescited by 101
  8. 2019Safe, Efficient, and Comfortable Velocity Control based on Reinforcement Learning for Autonomous Drivingcited by 56
  9. 2022Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Drivingcited by 34
  10. 2020A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learningcited by 24
  11. 2023EPG-MGCN: Ego-Planning Guided Multi-Graph Convolutional Network for Heterogeneous Agent Trajectory Predictioncited by 10
  12. 2021Development and testing of an image transformer for explainable autonomous driving systemscited by 9
  13. 2020Mixed platoon control of automated and human-driven vehicles at a signalized intersection: dynamical analysis and optimal controlcited by 6
  14. 2022Merging Control Strategies of Connected and Autonomous Vehicles at Freeway On-Ramps: A Comprehensive Reviewcited by 6
  15. 2023CV2X-LOCA: Roadside Unit-Enabled Cooperative Localization Framework for Autonomous Vehiclescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 13; 2026: 8.

31 papers.

Sort:

By area:

  1. 2024V2X-VLM: End-to-End V2X Cooperative Autonomous Driving Through Large Vision-Language Modelscited by 47
  2. 2025CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Modelscited by 11
  3. 2024A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedbackcited by 9
  4. 2024Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Drivingcited by 8
  5. 2025VLM-SAFE: Vision-Language Model-Guided Safety-Aware Reinforcement Learning with World Models for Autonomous Drivingcited by 8
  6. 2024Efficient and Generalized end-to-end Autonomous Driving System with Latent Deep Reinforcement Learning and Demonstrationscited by 7
  7. 2024Traffic expertise meets residual RL: Knowledge-informed model-based residual reinforcement learning for CAV trajectory controlcited by 7
  8. 2025Formalising Human-in-the-Loop: Computational Reductions, Failure Modes, and Legal-Moral Responsibilitycited by 7
  9. 2024Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunitiescited by 6
  10. 2025Knowledge Augmented Complex Problem Solving with Large Language Models: A Surveycited by 6
  11. 2025Beyond conventional vision: RGB-event fusion for robust object detection in dynamic traffic scenarioscited by 6
  12. 2024Knowledge Transfer from Simple to Complex: A Safe and Efficient Reinforcement Learning Framework for Autonomous Driving Decision-Makingcited by 5
19 more
  1. 2025SafePLUG: Empowering Multimodal LLMs with Pixel-Level Insight and Temporal Grounding for Traffic Accident Understandingcited by 5
  2. 2026HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Drivingcited by 5
  3. 2024MetaSSC: Enhancing 3D Semantic Scene Completion for Autonomous Driving through Meta-Learning and Long-sequence Modelingcited by 4
  4. 2024FollowGen: A Scaled Noise Conditional Diffusion Model for Car-Following Trajectory Predictioncited by 3
  5. 2025Data-Efficient Learning from Human Interventions for Mobile Robotscited by 3
  6. 2025Out-of-Distribution Detection for Safety Assurance of AI and Autonomous Systemscited by 2
  7. 2025T2IBias: Uncovering Societal Bias Encoded in the Latent Space of Text-to-Image Generative Modelscited by 2
  8. 2024Adaptive Decision-Making for Autonomous Vehicles: A Learning-Enhanced Game-Theoretic Approach in Interactive Environmentscited by 1
  9. 2025LaViPlan : Language-Guided Visual Path Planning with RLVRcited by 1
  10. 2025Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approachcited by 1
  11. 2025Sky-Drive: A Distributed Multi-Agent Simulation Platform for Human-AI Collaborative and Socially-Aware Future Transportationcited by 0
  12. 2025Confidence-Guided Human-AI Collaboration: Reinforcement Learning with Distributional Proxy Value Propagation for Autonomous Drivingcited by 0
  13. 2026Sparse Threats, Focused Defense: Criticality-Aware Robust Reinforcement Learning for Safe Autonomous Drivingcited by 0
  14. 2026Software-Hardware Co-optimization for Modular E2E AV Paradigm: A Unified Framework of Optimization Approaches, Simulation Environment and Evaluation Metricscited by 0
  15. 2026Risk-Aware Human-in-the-Loop Framework with Adaptive Intrusion Response for Autonomous Vehiclescited by 0
  16. 2026Bridging Policy and Real-World Dynamics: LLM-Augmented Rebalancing for Shared Micromobility Systemscited by 0
  17. 2026Sim2Real-AD: A Modular Sim-to-Real Framework for Deploying VLM-Guided Reinforcement Learning in Real-World Autonomous Drivingcited by 0
  18. 2026Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Trafficcited by 0
  19. 2026Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Human-in-the-Loop Deep Reinforcement Learning with Application to Autonomous Driving
  2. 2025From Learning to Mastery: Achieving Safe and Efficient Real-World Autonomous Driving with Human-In-The-Loop Reinforcement Learning
  3. 2024Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHF
  4. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
  5. 2021Modified DDPG car-following model with a real-world human driving experience with CARLA simulator
  6. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
  7. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving
  8. 2026Safe Reinforcement Learning of Autonomous Highway Driving: A Unified Framework for Safety and Efficiency