mlsanityData to 14 Oct 2026

Human-Centric Autonomous Systems With LLMs for User Command Reasoning

arXiv 2311.08206 ↗First posted 14 Nov 2023cs.CL cs.AI cs.ROcited by 22

On the map: LLM reasoning → Chain-of-thought reasoning for driving

Abstract (arXiv)

The evolution of autonomous driving has made remarkable advancements in recent years, evolving into a tangible reality. However, a human-centric large-scale adoption hinges on meeting a variety of multifaceted requirements. To ensure that the autonomous system meets the user's intent, it is essential to accurately discern and interpret user commands, especially in complex or emergency situations. To this end, we propose to leverage the reasoning capabilities of Large Language Models (LLMs) to infer system requirements from in-cabin users' commands. Through a series of experiments that include different LLM models and prompt designs, we explore the few-shot multivariate binary classification accuracy of system requirements from natural language textual commands. We confirm the general ability of LLMs to understand and reason about prompts but underline that their effectiveness is conditioned on the quality of both the LLM model and the design of appropriate sequential prompts. Code and models are public with the link urlhttps://github.com/KTH-RPL/DriveCmdLLM.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 14 of its 46 references that are in the map. First posted on arXiv 14 Nov 2023.

Builds on

14 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles12 Oct
  2. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Models14 Jul

Other references · 12

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2022Large Language Models Are Reasoning Teacherscited by 329
  7. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  8. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 126
  9. 2023Language Prompt for Autonomous Drivingcited by 111
  10. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehiclescited by 100
  11. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  12. 2023HiLM-D: Enhancing MLLMs with Multi-Scale High-Resolution Details for Autonomous Drivingcited by 50

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 11; 2025: 5; 2026: 4.

22 papers.

Sort:

By area:

  1. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  2. 2024AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attackscited by 96
  3. 2024A Survey for Foundation Models in Autonomous Drivingcited by 52
  4. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programscited by 45
  5. 2024Probing Multimodal LLMs as World Models for Drivingcited by 20
  6. 2025Towards Human-Centric Autonomous Driving: A Fast-Slow Architecture Integrating Large Language Model Guidance with Reinforcement Learningcited by 11
  7. 2026Bench2Drive-VL: Benchmarks for Closed-Loop Autonomous Driving with Vision-Language Modelscited by 10
  8. 2025From Text to Trust: Empowering AI-assisted Decision Making with Adaptive LLM-powered Analysiscited by 9
  9. 2024A Survey on Human-AI Collaboration with Large Foundation Modelscited by 7
  10. 2024XLM for Autonomous Driving Systems: A Comprehensive Reviewcited by 5
  11. 2024From Words to Wheels: Automated Style-Customized Policy Generation for Autonomous Drivingcited by 5
  12. 2024Hard Cases Detection in Motion Prediction by Vision-Language Foundation Modelscited by 4
10 more
  1. 2024A Survey on Large Language Model-empowered Autonomous Drivingcited by 4
  2. 2024TalkWithMachines: Enhancing Human-Robot Interaction for Interpretable Industrial Robotics Through Large/Vision Language Modelscited by 4
  3. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challengescited by 3
  4. 2024JMI at SemEval 2024 Task 3: Two-step approach for multimodal ECAC using in-context learning with GPT and instruction-tuned Llama modelscited by 2
  5. 2025Sage Deer: A Super-Aligned Driving Generalist Is Your Copilotcited by 2
  6. 2025E3AD: An Emotion-Aware Vision-Language-Action Model for Human-Centric End-to-End Autonomous Drivingcited by 2
  7. 2024Traj-LLM: A New Exploration for Empowering Trajectory Prediction with Pre-trained Large Language Modelscited by 0
  8. 2026A Large-Language-Model Supported Personalized Driving Framework for Lane Change in Highway Scenarioscited by 0
  9. 2026Talk to Me, Jarvis: An Open-Source Edge-Deployable Voice Assistant Framework for Autonomous Racecarscited by 0
  10. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platformcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving
  2. 2023Personalized Autonomous Driving with Large Language Models: Field Experiments
  3. 2026Modular Autonomy with Conversational Interaction: An LLM-driven Framework for Decision Making in Autonomous Driving
  4. 2024Hybrid Reasoning Based on Large Language Models for Autonomous Car Driving
  5. 2024Large Language Models for Human-like Autonomous Driving: A Survey
  6. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving
  7. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
  8. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Driving