mlsanityData to 14 Oct 2026

SurrealDriver: Designing LLM-powered Generative Driver Agent Framework based on Human Drivers' Driving-thinking Data

arXiv 2309.13193 ↗First posted 22 Sep 2023cs.HCcited by 24

On the map: LLM reasoning → Chain-of-thought reasoning for driving

also in 1 more
Abstract (arXiv)

Leveraging advanced reasoning capabilities and extensive world knowledge of large language models (LLMs) to construct generative agents for solving complex real-world problems is a major trend. However, LLMs inherently lack embodiment as humans, resulting in suboptimal performance in many embodied decision-making tasks. In this paper, we introduce a framework for building human-like generative driving agents using post-driving self-report driving-thinking data from human drivers as both demonstration and feedback. To capture high-quality, natural language data from drivers, we conducted urban driving experiments, recording drivers' verbalized thoughts under various conditions to serve as chain-of-thought prompts and demonstration examples for the LLM-Agent. The framework's effectiveness was evaluated through simulations and human assessments. Results indicate that incorporating expert demonstration data significantly reduced collision rates by 81.04% and increased human likeness by 50% compared to a baseline LLM-based agent. Our study provides insights into using natural language-based human demonstration data for embodied tasks. The driving-thinking dataset is available at urlhttps://github.com/AIR-DISCOVER/Driving-Thinking-Dataset.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 18 of its 24 references that are in the map. First posted on arXiv 22 Sep 2023.

Builds on

18 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Models14 Jul

Other references · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  6. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  7. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  8. 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
  9. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  10. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 462
  11. 2023Role-Play with Large Language Modelscited by 294
  12. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehiclescited by 100
5 more
  1. 2023Embodied Task Planning with Large Language Modelscited by 84
  2. 2023Dr.ICL: Demonstration-Retrieved In-context Learningcited by 72
  3. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 62
  4. 2023SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environmentscited by 53
  5. 2023Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agentscited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 17; 2026: 4.

24 papers.

Sort:

By area:

  1. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
  2. 2025Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advancescited by 21
  3. 2025DebFlow: Automating Agent Creation via Agent Debatecited by 10
  4. 2025WaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Drivingcited by 9
  5. 2025Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Reviewcited by 8
  6. 2025LiloDriver: A Lifelong Learning Framework for Closed-loop Motion Planning in Long-tail Autonomous Driving Scenarioscited by 7
  7. 2024A Survey on Large Language Model-empowered Autonomous Drivingcited by 4
  8. 2025QuadSentinel: Sequent Safety for Machine-Checkable Control in Multi-agent Systemscited by 4
  9. 2025Carbon and Silicon, Coexist or Compete? A Survey on Human-AI Interactions in Agent-based Modeling and Simulationcited by 3
  10. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challengescited by 3
  11. 2026CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systemscited by 3
  12. 2025GeoVLM: Improving Automated Vehicle Geolocalisation Using Vision-Language Matchingcited by 2
12 more
  1. 2025Predicting Effects, Missing Distributions: Evaluating LLMs as Human Behavior Simulators in Operations Managementcited by 2
  2. 2025Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulationscited by 2
  3. 2025Multimodal Large Language Models for Enhanced Traffic Safety: A Comprehensive Review and Future Trendscited by 1
  4. 2025A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approachcited by 1
  5. 2024A Novel MLLM-based Approach for Autonomous Driving in Different Weather Conditionscited by 0
  6. 2025LimSim Series: An Autonomous Driving Simulation Platform for Validation and Enhancementcited by 0
  7. 2025Sky-Drive: A Distributed Multi-Agent Simulation Platform for Human-AI Collaborative and Socially-Aware Future Transportationcited by 0
  8. 2025Understanding Driving Risks using Large Language Models: Toward Elderly Driver Assessmentcited by 0
  9. 2025SocialDriveGen: Generating Diverse Traffic Scenarios with Controllable Social Interactionscited by 0
  10. 2026Language-Driven Cost Optimization for Autonomous Drivingcited by 0
  11. 2026Vision-Language Assistant for Emotional Reactions to Risky Drivingcited by 0
  12. 2026Are LLMs Good Financial User Simulators? Multi-view Investor Logic Alignment (MILA)cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Driving Style Alignment for LLM-powered Driver Agent
  2. 2023A Language Agent for Autonomous Driving
  3. 2023Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles
  4. 2025AGENTS-LLM: Augmentative GENeration of Challenging Traffic Scenarios with an Agentic LLM Framework
  5. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
  6. 2023Personalized Autonomous Driving with Large Language Models: Field Experiments
  7. 2024Large Language Models for Human-like Autonomous Driving: A Survey
  8. 2024Text-to-Drive: Diverse Driving Behavior Synthesis via Large Language Models