mlsanityData to 7 Oct 2026

Driving Everywhere with Large Language Model Policy Adaptation

arXiv 2402.05932 ↗First posted 8 Feb 2024cs.RO cs.AI cs.CLcited by 53

On the map: End-to-end autonomous driving → Large language models for driving

Abstract (arXiv)

Adapting driving behavior to new environments, customs, and laws is a long-standing problem in autonomous driving, precluding the widespread deployment of autonomous vehicles (AVs). In this paper, we present LLaDA, a simple yet powerful tool that enables human drivers and autonomous vehicles alike to drive everywhere by adapting their tasks and motion plans to traffic rules in new locations. LLaDA achieves this by leveraging the impressive zero-shot generalizability of large language models (LLMs) in interpreting the traffic rules in the local driver handbook. Through an extensive user study, we show that LLaDA's instructions are useful in disambiguating in-the-wild unexpected situations. We also demonstrate LLaDA's ability to adapt AV motion planning policies in real-world datasets; LLaDA outperforms baseline planning approaches on all our metrics. Please check our website for more details: https://boyiliee.github.io/llada.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving. Builds on 22 of its 35 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

22 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct

Other references · 21

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  3. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  4. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  5. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  6. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  7. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  8. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  9. 2021VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehiclescited by 81
  10. 2023Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Modelscited by 59
  11. 2023Semantic Anomaly Detection with Large Language Modelscited by 53
  12. 2023Interactive Task Planning with Language Modelscited by 36
9 more
  1. 2015Plan Explicability and Predictability for Robot Task Planningcited by 26
  2. 2022Expanding the Deployment Envelope of Behavior Prediction via Adaptive Meta-Learningcited by 23
  3. 2022Interpretable Self-Aware Neural Networks for Robust Trajectory Predictioncited by 22
  4. 2021Rule-based Optimal Control for Autonomous Drivingcited by 21
  5. 2018Robustness to Out-of-Distribution Inputs via Task-Aware Generative Uncertaintycited by 14
  6. 2020Formalizing Traffic Rules for Machine Interpretabilitycited by 12
  7. 2022Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehiclescited by 11
  8. 2022Receding Horizon Planning with Rule Hierarchies for Autonomous Vehiclescited by 7
  9. 2023Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictorscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 24; 2026: 14.

53 papers.

Sort:

By area:

  1. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  2. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  3. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  4. 2024Tokenize the World into Object-level Knowledge to Address Long-tail Events in Autonomous Drivingcited by 55
  5. 2024VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Modelscited by 46
  6. 2024PlanAgent: A Multi-modal Large Language Agent for Closed-loop Vehicle Motion Planningcited by 38
  7. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  8. 2025Adversarial Attacks on Robotic Vision Language Action Modelscited by 31
  9. 2025Safety Guardrails for LLM-Enabled Robotscited by 30
  10. 2025V2V-LLM: Vehicle-to-Vehicle Cooperative Autonomous Driving with Multimodal Large Language Modelscited by 27
  11. 2024Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systemscited by 23
  12. 2024VLM-Auto: VLM-based Autonomous Driving Assistant with Human-like Behavior and Understanding for Complex Road Scenescited by 22
38 more
  1. 2024ROADWork: A Dataset and Benchmark for Learning to Recognize, Observe, Analyze and Drive Through Work Zonescited by 20
  2. 2025Rethinking Progression of Memory State in Robotic Manipulation: An Object-Centric Perspectivecited by 20
  3. 2024AD-H: Language-guided Autonomous Driving with Hierarchical Agentscited by 15
  4. 2024Driving with Regulation: Trustworthy and Interpretable Decision-Making for Autonomous Driving with Retrieval-Augmented Reasoningcited by 11
  5. 2026Learning Vision-Language-Action World Models for Autonomous Drivingcited by 11
  6. 2024LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trendscited by 10
  7. 2025Distilling On-device Language Models for Robot Planning with Minimal Human Interventioncited by 10
  8. 2025DriveQA: Passing the Driving Knowledge Testcited by 10
  9. 2025UniUGP: Unifying Understanding, Generation, and Planing For End-to-end Autonomous Drivingcited by 10
  10. 2026Unifying Language-Action Understanding and Generation for Autonomous Drivingcited by 9
  11. 2026nuReasoning: A Reasoning-Centric Dataset and Benchmark for Long-Tail Autonomous Drivingcited by 8
  12. 2024Wolf: Dense Video Captioning with a World Summarization Frameworkcited by 7
  13. 2025Tracking Meets Large Multimodal Models for Driving Scenario Understandingcited by 7
  14. 2025Learning to Tune Like an Expert: Interpretable and Scene-Aware Navigation via MLLM Reasoning and CVAE-Based Adaptationcited by 6
  15. 2026Learning to Drive in New Cities Without Human Demonstrationscited by 6
  16. 2024TraveLLM: Could you plan my new public transit route in face of a network disruption?cited by 5
  17. 2025Planning Safety Trajectories with Dual-Phase, Physics-Informed, and Transportation Knowledge-Driven Large Language Modelscited by 5
  18. 2025LeAD: The LLM Enhanced Planning System Converged with End-to-end Autonomous Drivingcited by 5
  19. 2025Tailored Primitive Initialization is the Secret Key to Reinforcement Learningcited by 4
  20. 2024A Survey on Large Language Model-empowered Autonomous Drivingcited by 3
  21. 2024MTA: Multimodal Task Alignment for BEV Perception and Captioningcited by 3
  22. 2025Deploying Foundation Model-Enabled Air and Ground Robots in the Field: Challenges and Opportunitiescited by 3
  23. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challengescited by 3
  24. 2024RuleFuser: An Evidential Bayes Approach for Rule Injection in Imitation Learned Planners and Predictors for Robustness under Distribution Shiftscited by 2
  25. 2024How Could Generative AI Support Compliance with the EU AI Act? A Review for Safe Automated Driving Perceptioncited by 2
  26. 2025Knowledge Integration Strategies in Autonomous Vehicle Prediction and Planning: A Comprehensive Surveycited by 1
  27. 2025CorrA: Leveraging Large Language Models for Dynamic Obstacle Avoidance of Autonomous Vehiclescited by 1
  28. 2025From Refusal to Recovery: A Control-Theoretic Approach to Generative AI Guardrailscited by 1
  29. 2025Synergy over Discrepancy: A Partition-Based Approach to Multi-Domain LLM Fine-Tuningcited by 1
  30. 2026VLGOR: Visual-Language Knowledge Guided Offline Reinforcement Learning for Generalizable Agentscited by 1
  31. 2026Pedestrian-Aware LLM-Driven Behavioral Planning for Autonomous Vehiclescited by 1
  32. 2026D3VL: Understanding Driving Scenes from 3D Time Series Data and Video with Language Modelscited by 1
  33. 2025Strategic Fusion of Vision Language Models: Shapley-Credited Context-Aware Dawid-Skene for Multi-Label Tasks in Autonomous Drivingcited by 0
  34. 2025A Low-Rank Method for Vision Language Model Hallucination Mitigation in Autonomous Drivingcited by 0
  35. 2026The Constant Eye: Benchmarking and Bridging Appearance Robustness in Autonomous Drivingcited by 0
  36. 2026Test-Driven Agentic Framework for Reliable Robot Controllercited by 0
  37. 2026Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Supportcited by 0
  38. 2026PixelPilot: Scalable Vision-Language-Action Models for End-to-End Autonomous Drivingcited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Driving
  2. 2024Text-to-Drive: Diverse Driving Behavior Synthesis via Large Language Models
  3. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
  4. 2024Generalizing End-To-End Autonomous Driving In Real-World Environments Using Zero-Shot LLMs
  5. 2023A Survey on Multimodal Large Language Models for Autonomous Driving
  6. 2025AdaDrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous Driving
  7. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving
  8. 2024Probing Multimodal LLMs as World Models for Driving