Align2Act: Instruction-Tuned Models for Human-Aligned Autonomous Driving
On the map: End-to-end autonomous driving → Reasoning VLA and VLM driving policies
Abstract (arXiv)
Motion planning in complex scenarios is a core challenge in autonomous driving. Conventional methods apply predefined rules or learn from driving data to generate trajectories, while recent approaches leverage large language models (LLMs) for decision-making. However, it remains unclear whether LLMs truly capture human driving logic. We propose Align2Act, a motion planning framework that transforms instruction-tuned LLMs into interpretable planners aligned with human behavior. We derive structured driving instructions based on human reasoning patterns (e.g., anticipate hazards, yield at intersections) and traffic rules (e.g., stop at red lights, maintain lane boundaries). Our Align2ActChain module guides step-by-step reasoning to produce both an interpretable rationale and a safe trajectory. By fine-tuning LLaMA-2-7B with LoRA on one million scenarios from the nuPlan dataset, our method achieves an open-loop score of 85.17 and closed-loop scores of 70.31 (non-reactive) and 66.96 (reactive) on Test14-random. Unlike prior work focused on synthetic or open-loop settings, we demonstrate improved planning quality and human-likeness on the real-world nuPlan closed-loop benchmark. Ablation studies confirm that structured reasoning significantly improves performance over baseline LLM planners.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning VLA and VLM driving policies, one of 9 directions in End-to-end autonomous driving. Builds on 27 of its 30 references that are in the map. First posted on arXiv 12 Oct 2025.
Builds on
27 of 30 references.
References · 27
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2016End to End Learning for Self-Driving Carscited by 1,000
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
- 2017End-to-end Driving via Conditional Imitation Learningcited by 479
- 2023GPT-Driver: Learning to Drive with GPTcited by 344
- 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
- 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
15 more
- 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
- 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
- 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
- 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
- 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
- 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
- 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
- 2024Holistic Autonomous Driving Understanding by Bird's-Eye-View Injected Multi-Modal Large Modelscited by 75
- 2023LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoningcited by 59
- 2023From Prediction to Planning With Goal Conditioned Lane Graph Traversalscited by 45
- 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
- 2024Instruct Large Language Models to Drive like Humanscited by 11
- 2024Fine-Tuning Gemma-7B for Enhanced Sentiment Analysis of Financial News Headlinescited by 8
- 2024Driving with InternVL: Oustanding Champion in the Track on Driving with Language of the Autonomous Grand Challenge at CVPR 2024cited by 5
- 2025The System Description of CPS Team for Track on Driving with Language of CVPR 2024 Autonomous Grand Challengecited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AlignDrive: Aligned Lateral-Longitudinal Planning for End-to-End Autonomous Driving
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
- 2024Driving Style Alignment for LLM-powered Driver Agent
- 2025Plan-R1: Safe and Feasible Trajectory Planning as Language Modeling
- 2025DriveCritic: Towards Context-Aware, Human-Aligned Evaluation for Autonomous Driving with Vision-Language Models
- 2025ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Driving
- 2025SAFe-Copilot: Unified Shared Autonomy Framework ↗
- 2026DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving