mlsanityData to 7 Oct 2026

Reason--Imagine--Act: Closed-Loop LLM Decision Making with World Models for Autonomous Driving

arXiv 2605.24004 ↗First posted 19 May 2026cs.AI cs.CV cs.LGcited by 0

On the map: World models → World models for autonomous driving

also in 2 more
Abstract (arXiv)

Large language models (LLMs) are promising for autonomous driving, but semantics-only decision policies can yield physically unsafe behavior in dynamic traffic. Existing methods either perform online language reasoning without explicit dynamics verification or use world models mainly in offline pipelines, leaving a gap between semantic intent and physical feasibility at decision time. We propose Reason--Imagine--Act (RIA), a closed-loop framework that couples an LLM reasoner with an action-conditioned world model for online safety verification. At each step, the LLM proposes an action template and candidate sub-actions, the world model performs short-horizon rollouts, and a safety scorer selects the safest executable action with feedback to the next reasoning step. Under a unified CARLA point-goal protocol (1000 episodes), RIA achieves 80.05% route completion, 51.10% arrival rate, and 0.20% collision rate. Under the same closed-loop interface, RIA consistently outperforms training-free baselines, including CARLA TM and MADA, on core closed-loop metrics. For reproducibility, code is available at https://github.com/pku-smart-city/source_code/tree/main/RIA.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 19 of its 25 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

19 of 25 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025AdaWM: Adaptive World Model based Planning for Autonomous Driving22 Jan
  2. 2024Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving26 Aug
  3. 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
  4. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving27 Nov
  5. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 14

  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  2. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  3. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  4. 2019Learning by Cheatingcited by 361
  5. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  6. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  7. 2023DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Modelscited by 204
  8. 2023Personalized Autonomous Driving with Large Language Models: Field Experimentscited by 46
  9. 2025Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Modelscited by 27
  10. 2023FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modelingcited by 21
  11. 2025SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Drivingcited by 21
  12. 2024Driving Style Alignment for LLM-powered Driver Agentcited by 15
2 more
  1. 2024Large Language Models for Human-like Autonomous Driving: A Surveycited by 14
  2. 2024MSMA: Multi-agent Trajectory Prediction in Connected and Autonomous Vehicle Environment with Multi-source Data Integrationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving
  2. 2025SanDRA: Safe Large-Language-Model-Based Decision Making for Automated Vehicles Using Reachability Analysis
  3. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving
  4. 2026ReasonBreak: Probing Vulnerabilities in Reasoning-Enabled Vision-Language-Action Models for Autonomous Driving
  5. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Driving
  6. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs
  7. 2026Orion-Lite: Distilling LLM Reasoning into Efficient Vision-Only Driving Models
  8. 2026Is VLA Reasoning Faithful? Probing Safety of Chain-of-Causation in Autonomous Driving Models