mlsanityData to 7 Oct 2026

Autoware.Flex: Human-Instructed Dynamically Reconfigurable Autonomous Driving Systems

arXiv 2412.16265 ↗First posted 20 Dec 2024cs.AI cs.HC cs.ROcited by 3

On the map: End-to-end autonomous driving → Large language models for driving

Abstract (arXiv)

Existing Autonomous Driving Systems (ADS) independently make driving decisions, but they face two significant limitations. First, in complex scenarios, ADS may misinterpret the environment and make inappropriate driving decisions. Second, these systems are unable to incorporate human driving preferences in their decision-making processes. This paper proposes Autoware.Flex, a novel ADS system that incorporates human input into the driving process, allowing users to guide the ADS in making more appropriate decisions and ensuring their preferences are satisfied. Achieving this needs to address two key challenges: (1) translating human instructions, expressed in natural language, into a format the ADS can understand, and (2) ensuring these instructions are executed safely and consistently within the ADS' s decision-making framework. For the first challenge, we employ a Large Language Model (LLM) assisted by an ADS-specialized knowledge base to enhance domain-specific translation. For the second challenge, we design a validation mechanism to ensure that human instructions result in safe and consistent driving behavior. Experiments conducted on both simulators and a real-world autonomous vehicle demonstrate that Autoware.Flex effectively interprets human instructions and executes them safely.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving. Builds on 23 of its 42 references that are in the map. First posted on arXiv 20 Dec 2024.

Builds on

23 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Generalizing End-To-End Autonomous Driving In Real-World Environments Using Zero-Shot LLMs21 Nov
  2. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Models12 Dec
  3. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Driving2 Nov
  4. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving3 Oct
  5. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct
  6. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Models14 Jul

Other references · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
  5. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  6. 2022Robot Operating System 2: Design, Architecture, and Uses In The Wildcited by 190
  7. 2019Talk2Car: Taking Control of Your Self-Driving Carcited by 141
  8. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  9. 2018Scenic: A Language for Scenario Specification and Scene Generationcited by 79
  10. 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoningcited by 59
  11. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
  12. 2024Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domainscited by 30
5 more
  1. 2024T-RAG: Lessons from the LLM Trenchescited by 23
  2. 2022LawBreaker: An Approach for Specifying Traffic Laws and Fuzzing Autonomous Vehiclescited by 15
  3. 2021Safety of the Intended Driving Behavior Using Rulebookscited by 8
  4. 2024REDriver: Runtime Enforcement for Autonomous Vehiclescited by 5
  5. 2022Winning the 3rd Japan Automotive AI Challenge -- Autonomous Racing with the Autoware.Auto Open Source Software Stackcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025COVLM-RL: Critical Object-Oriented Reasoning for Autonomous Driving Using VLM-Guided Reinforcement Learningcited by 1
  2. 2026Modular Autonomy with Conversational Interaction: An LLM-driven Framework for Decision Making in Autonomous Drivingcited by 1
  3. 2026A Large-Language-Model Supported Personalized Driving Framework for Lane Change in Highway Scenarioscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Hint-AD: Holistically Aligned Interpretability in End-to-End Autonomous Driving
  2. 2025Towards Human-Centric Autonomous Driving: A Fast-Slow Architecture Integrating Large Language Model Guidance with Reinforcement Learning
  3. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs
  4. 2025Align2Act: Instruction-Tuned Models for Human-Aligned Autonomous Driving
  5. 2024Large Language Models for Human-like Autonomous Driving: A Survey
  6. 2023Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles
  7. 2021Explainable Artificial Intelligence for Autonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions
  8. 2019Conditional Driving from Natural Language Instructions