# Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHF

arXiv: https://arxiv.org/abs/2406.04481 · HTML: https://mlsanity.com/paper/2406.04481/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 6 Jun 2024.

## Placed in

- End-to-end autonomous driving / [Large language models for driving](https://mlsanity.com/driving/large-language-models-for-driving/)
- LLM alignment / RLHF / preference optimisation / [Reward modeling and RLHF](https://mlsanity.com/alignment/reward-modeling-and-rlhf/)

## Builds on (most cited first, 12 of 12)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisons](https://mlsanity.com/paper/2301.11270/) (2023)
- [Language Conditioned Traffic Generation](https://mlsanity.com/paper/2307.07947/) (2023)
- [Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning](https://mlsanity.com/paper/1803.10056/) (2018)
- [Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors](https://mlsanity.com/paper/2305.04412/) (2023)
- [LimSim++: A Closed-Loop Platform for Deploying Multimodal LLMs in Autonomous Driving](https://mlsanity.com/paper/2402.01246/) (2024)
- [Reinforcement Learning in the Era of LLMs: What is Essential? What is needed? An RL Perspective on RLHF, Prompting, and Beyond](https://mlsanity.com/paper/2310.06147/) (2023)
- [Balancing Enhancement, Harmlessness, and General Capabilities: Enhancing Conversational LLMs with Direct RLHF](https://mlsanity.com/paper/2403.02513/) (2024)
- [Prompting Multi-Modal Tokens to Enhance End-to-End Autonomous Driving Imitation Learning with LLMs](https://mlsanity.com/paper/2404.04869/) (2024)

## Built on by (most cited first, 18 of 18)

- [G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems](https://mlsanity.com/paper/2506.07398/) (2025)
- [Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions](https://mlsanity.com/paper/2501.04437/) (2025)
- [Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances](https://mlsanity.com/paper/2502.16804/) (2025)
- [Learning Personalized Driving Styles via Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2503.10434/) (2025)
- [LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trends](https://mlsanity.com/paper/2410.15281/) (2024)
- [SafeDrive: Knowledge- and Data-Driven Risk-Sensitive Decision-Making for Autonomous Vehicles with Large Language Models](https://mlsanity.com/paper/2412.13238/) (2024)
- [Next-Generation LLM for UAV: From Natural Language to Autonomous Flight](https://mlsanity.com/paper/2510.21739/) (2025)
- [A Comprehensive LLM-powered Framework for Driving Intelligence Evaluation](https://mlsanity.com/paper/2503.05164/) (2025)
- [HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving](https://mlsanity.com/paper/2505.15793/) (2025)
- [Leveraging Large Language Models for Enhancing Autonomous Vehicle Perception](https://mlsanity.com/paper/2412.20230/) (2024)
- [Understanding the Effects of RLHF on the Quality and Detectability of LLM-Generated Texts](https://mlsanity.com/paper/2503.17965/) (2025)
- [Post-Training in End-to-End Autonomous Driving](https://mlsanity.com/paper/2607.08072/) (2026)
- [Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach](https://mlsanity.com/paper/2510.09041/) (2025)
- [Regret Bounds for Reinforcement Learning from Multi-Source Imperfect Preferences](https://mlsanity.com/paper/2603.20453/) (2026)
- [SARAD: LLM-Based Safety-Aware Hybrid Reinforcement Learning with Collision Prediction for Autonomous Driving](https://mlsanity.com/paper/2605.28583/) (2026)
- [The Streetscape Application Services Stack (SASS): Towards a Distributed Sensing Architecture for Urban Applications](https://mlsanity.com/paper/2411.19714/) (2024)
- [LA-RL: Language Action-guided Reinforcement Learning with Safety Guarantees for Autonomous Highway Driving](https://mlsanity.com/paper/2512.05686/) (2025)
- [Taming Asynchronous CPU-GPU Coupling for Frequency-aware Latency Estimation on Mobile Edge](https://mlsanity.com/paper/2604.15357/) (2026)

## Abstract (arXiv)

Reinforcement Learning from Human Feedback (RLHF) is popular in large language models (LLMs), whereas traditional Reinforcement Learning (RL) often falls short. Current autonomous driving methods typically utilize either human feedback in machine learning, including RL, or LLMs. Most feedback guides the car agent's learning process (e.g., controlling the car). RLHF is usually applied in the fine-tuning step, requiring direct human "preferences," which are not commonly used in optimizing autonomous driving models. In this research, we innovatively combine RLHF and LLMs to enhance autonomous driving safety. Training a model with human guidance from scratch is inefficient. Our framework starts with a pre-trained autonomous car agent model and implements multiple human-controlled agents, such as cars and pedestrians, to simulate real-life road environments. The autonomous car model is not directly controlled by humans. We integrate both physical and physiological feedback to fine-tune the model, optimizing this process using LLMs. This multi-agent interactive environment ensures safe, realistic interactions before real-world application. Finally, we will validate our model using data gathered from real-life testbeds located in New Jersey and New York City.