# Human-In-The-Loop Machine Learning for Safe and Ethical Autonomous Vehicles: Principles, Challenges, and Opportunities

arXiv: https://arxiv.org/abs/2408.12548 · HTML: https://mlsanity.com/paper/2408.12548/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 29 of its 111 references that are in the map.
- First posted on arXiv 22 Aug 2024.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 29 of 29)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [A Survey of Human-in-the-loop for Machine Learning](https://mlsanity.com/paper/2108.00941/) (2021)
- [A Survey of Deep Learning Applications to Autonomous Vehicle Control](https://mlsanity.com/paper/1912.10773/) (2019)
- [Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey](https://mlsanity.com/paper/2307.04370/) (2023)
- [A Survey on Interactive Reinforcement Learning: Design Principles and Open Challenges](https://mlsanity.com/paper/2105.12949/) (2021)
- [HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving](https://mlsanity.com/paper/2401.03160/) (2024)
- [Machine Learning-Aided Operations and Communications of Unmanned Aerial Vehicles: A Contemporary Survey](https://mlsanity.com/paper/2211.04324/) (2022)
- [LLM A*: Human in the Loop Large Language Models Enabled A* Search for Robotics](https://mlsanity.com/paper/2312.01797/) (2023)
- [Agent-Agnostic Human-in-the-Loop Reinforcement Learning](https://mlsanity.com/paper/1701.04079/) (2017)
- [Prioritized Experience-based Reinforcement Learning with Human Guidance for Autonomous Driving](https://mlsanity.com/paper/2109.12516/) (2021)
- [When Internet of Things meets Metaverse: Convergence of Physical and Cyber Worlds](https://mlsanity.com/paper/2208.13501/) (2022)
- [Exploring Diversity-based Active Learning for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/2205.07708/) (2022)
- [A Computational Model of Commonsense Moral Decision Making](https://mlsanity.com/paper/1801.04346/) (2018)
- [Reward-Driven Automated Curriculum Learning for Interaction-Aware Self-Driving at Unsignalized Intersections](https://mlsanity.com/paper/2403.13674/) (2024)
- [Collaborative Target Search with a Visual Drone Swarm: An Adaptive Curriculum Embedded Multistage Reinforcement Learning Approach](https://mlsanity.com/paper/2204.12181/) (2022)
- [An Informative Path Planning Framework for Active Learning in UAV-based Semantic Mapping](https://mlsanity.com/paper/2302.03347/) (2023)
- [LLMAuditor: A Framework for Auditing Large Language Models Using Human-in-the-Loop](https://mlsanity.com/paper/2402.09346/) (2024)
- [Interactive Machine Learning: A State of the Art Review](https://mlsanity.com/paper/2207.06196/) (2022)
- [Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle](https://mlsanity.com/paper/2001.03359/) (2020)
- [Conversational Language Models for Human-in-the-Loop Multi-Robot Coordination](https://mlsanity.com/paper/2402.19166/) (2024)
- [Scenario-Based Curriculum Generation for Multi-Agent Autonomous Driving](https://mlsanity.com/paper/2403.17805/) (2024)
- [Design and Implementation of Secret Key Agreement for Platoon-based Vehicular Cyber-Physical Systems](https://mlsanity.com/paper/1911.07617/) (2019)
- [Model-Driven Requirements for Humans-on-the-Loop Multi-UAV Missions](https://mlsanity.com/paper/2009.10267/) (2020)
- [Indoor Path Planning for an Unmanned Aerial Vehicle via Curriculum Learning](https://mlsanity.com/paper/2108.09986/) (2021)
- [A Brief Guide to Designing and Evaluating Human-Centered Interactive Machine Learning](https://mlsanity.com/paper/2204.09622/) (2022)
- [LCD: Low Latency Command Dissemination for A Platoon of Vehicles](https://mlsanity.com/paper/1801.06153/) (2018)
- [UAV Obstacle Avoidance by Human-in-the-Loop Reinforcement in Arbitrary 3D Environment](https://mlsanity.com/paper/2304.05959/) (2023)

## Built on by (most cited first, 6 of 6)

- [UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs](https://mlsanity.com/paper/2506.08045/) (2025)
- [AI and Human Oversight: A Risk-Based Framework for Alignment](https://mlsanity.com/paper/2510.09090/) (2025)
- [Large Language Model-Assisted UAV Operations and Communications: A Multifaceted Survey and Tutorial](https://mlsanity.com/paper/2602.19534/) (2026)
- [AdaptBot: Combining LLM with Knowledge Graphs and Human Input for Generic-to-Specific Task Decomposition and Knowledge Refinement](https://mlsanity.com/paper/2502.02067/) (2025)
- [Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems](https://mlsanity.com/paper/2410.23472/) (2024)
- [The SMART+ Framework for AI Systems](https://mlsanity.com/paper/2512.08592/) (2025)

## Abstract (arXiv)

Machine Learning (ML) has become central to Autonomous Vehicles (AVs), supporting perception, prediction, planning, control, and decision-making in dynamic environments. However, achieving full autonomy in cluttered and complex scenarios, such as intricate intersections, diverse scenes, varied trajectories, and complex missions, remains challenging; moreover, data labeling is still a major bottleneck. These limitations motivate Human-in-the-Loop Machine Learning (HITL-ML), in which human input is incorporated through validation, annotation, task organization, reward design, action correction, preference feedback, and supervisory intervention. To advance safe and ethical autonomy, this paper presents a tutorial survey of HITL-ML for AVs, focusing on Curriculum Learning (CL), Human-in-the-Loop Reinforcement Learning (HITL-RL), Human-in-the-Loop Large Language Models (HITL-LLMs), Active Learning (AL), and ethical principles. We first review CL methods that structure training from simple to complex tasks, covering navigation, path planning, obstacle avoidance, data collection, landing, intersection handling, motion planning, and UAV swarm coordination. We then examine HITL-RL through reward shaping, action injection, demonstrations, preference-based feedback, and interactive learning, emphasizing improved learning efficiency, safer policy exploration, and real-time intervention. Next, we review HITL-LLM through collaboration and oversight and specify key challenges. After that, we discuss AL for perception, anomaly detection, semantic mapping, object detection, vehicle recognition, and security-related classification. Ethical principles are reviewed as technical requirements for transparency, accountability, human oversight, safety, security, regulatory compliance, and reliability of human input.