mlsanityData to 7 Oct 2026

Language-Driven Cost Optimization for Autonomous Driving

arXiv 2606.10974 ↗First posted 9 Jun 2026cs.ROcited by 0

On the map: End-to-end autonomous driving → Large language models for driving

Abstract (arXiv)

The driving behavior of autonomous vehicles is typically governed by the cost function of their motion planner, which encodes objectives such as speed tracking, smoothness, lane keeping, and collision avoidance. However, tuning the parameters that shape this cost function is a challenging task that requires technical expertise, limiting the vehicle's ability to adapt to evolving traffic scenarios or end-user preferences. This work presents a language-driven framework for adaptive cost design in autonomous driving. A Large Language Model (LLM) interprets structured scenario descriptions and natural language user queries to generate the parameters applied to a risk-aware Model Predictive Path Integral (MPPI) controller. The system incorporates a human-in-the-loop validation stage in which the proposed behavioral changes are described in non-technical language and confirmed prior to deployment. Users may additionally provide feedback either before or after deployment, enabling iterative refinement of the vehicle's motion behavior. The framework is evaluated across multiple queries in realistic driving scenarios to assess its effectiveness. Simulation results demonstrate that the method successfully induces behavioral changes that align with the intended requirements in an intuitive manner, thereby bridging the gap between intelligent vehicle control systems and end users.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Large language models for driving, one of 9 directions in End-to-end autonomous driving. Builds on 31 of its 38 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

31 of 38 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Enhancing Autonomous Driving Systems with On-Board Deployed Large Language Models15 Apr
  2. 2024KoMA: Knowledge-driven Multi-agent Framework for Autonomous Driving with Large Language Models19 Jul
  3. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Models12 Dec
  4. 2023A Survey on Multimodal Large Language Models for Autonomous Driving21 Nov
  5. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct
  6. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models28 Sep
  7. 2023Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles19 Sep
  8. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Models14 Jul

Other references · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  8. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  9. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  12. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,081
11 more
  1. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  2. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  3. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 440
  4. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
  5. 2017Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Drivingcited by 81
  6. 2023SurrealDriver: Designing LLM-powered Generative Driver Agent Framework based on Human Drivers' Driving-thinking Datacited by 24
  7. 2023ChatMPC: Natural Language based MPC Personalizationcited by 5
  8. 2024From Words to Wheels: Automated Style-Customized Policy Generation for Autonomous Drivingcited by 5
  9. 2024Hey Robot! Personalizing Robot Navigation through Model Predictive Control with a Large Language Modelcited by 4
  10. 2025Dynamic Risk-Aware MPPI for Mobile Robots in Crowds via Efficient Monte Carlo Approximationscited by 2
  11. 2025SanDRA: Safe Large-Language-Model-Based Decision Making for Automated Vehicles Using Reachability Analysiscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving
  2. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
  3. 2023Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles
  4. 2023Empowering Autonomous Driving with Large Language Models: A Safety Perspective
  5. 2026Large Language Model based Interactive Decision-Making for Autonomous Driving
  6. 2025AdaDrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous Driving
  7. 2023Fine-Tuning Language Models Using Formal Methods Feedback
  8. 2025Foundation Models for Trajectory Planning in Autonomous Driving: A Review of Progress and Open Challenges