# Reason--Imagine--Act: Closed-Loop LLM Decision Making with World Models for Autonomous Driving

arXiv: https://arxiv.org/abs/2605.24004 · HTML: https://mlsanity.com/paper/2605.24004/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in World models for autonomous driving, one of 11 directions in World models.
- Builds on 19 of its 25 references that are in the map.
- First posted on arXiv 19 May 2026.

## Placed in

- World models / [World models for autonomous driving](https://mlsanity.com/wm/world-models-for-autonomous-driving/)
- LLM reasoning / [Chain-of-thought reasoning for driving](https://mlsanity.com/lr/chain-of-thought-reasoning-for-driving/)
- End-to-end autonomous driving / [Large language models for driving](https://mlsanity.com/driving/large-language-models-for-driving/)

## Builds on (most cited first, 19 of 19)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models](https://mlsanity.com/paper/2402.12289/) (2024)
- [DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model](https://mlsanity.com/paper/2310.01412/) (2023)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [LMDrive: Closed-Loop End-to-End Driving with Large Language Models](https://mlsanity.com/paper/2312.07488/) (2023)
- [AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.13757/) (2025)
- [OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving](https://mlsanity.com/paper/2311.16038/) (2023)
- [DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2309.16292/) (2023)
- [DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation](https://mlsanity.com/paper/2403.06845/) (2024)
- [Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving](https://mlsanity.com/paper/2408.14197/) (2024)
- [Personalized Autonomous Driving with Large Language Models: Field Experiments](https://mlsanity.com/paper/2312.09397/) (2023)
- [Occ-LLM: Enhancing Autonomous Driving with Occupancy-Based Large Language Models](https://mlsanity.com/paper/2502.06419/) (2025)
- [AdaWM: Adaptive World Model based Planning for Autonomous Driving](https://mlsanity.com/paper/2501.13072/) (2025)
- [FollowNet: A Comprehensive Benchmark for Car-Following Behavior Modeling](https://mlsanity.com/paper/2306.05381/) (2023)
- [SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Driving](https://mlsanity.com/paper/2505.16805/) (2025)
- [Driving Style Alignment for LLM-powered Driver Agent](https://mlsanity.com/paper/2403.11368/) (2024)
- [Large Language Models for Human-like Autonomous Driving: A Survey](https://mlsanity.com/paper/2407.19280/) (2024)
- [MSMA: Multi-agent Trajectory Prediction in Connected and Autonomous Vehicle Environment with Multi-source Data Integration](https://mlsanity.com/paper/2407.21310/) (2024)

## Abstract (arXiv)

Large language models (LLMs) are promising for autonomous driving, but semantics-only decision policies can yield physically unsafe behavior in dynamic traffic. Existing methods either perform online language reasoning without explicit dynamics verification or use world models mainly in offline pipelines, leaving a gap between semantic intent and physical feasibility at decision time. We propose Reason--Imagine--Act (RIA), a closed-loop framework that couples an LLM reasoner with an action-conditioned world model for online safety verification. At each step, the LLM proposes an action template and candidate sub-actions, the world model performs short-horizon rollouts, and a safety scorer selects the safest executable action with feedback to the next reasoning step. Under a unified CARLA point-goal protocol (1000 episodes), RIA achieves 80.05% route completion, 51.10% arrival rate, and 0.20% collision rate. Under the same closed-loop interface, RIA consistently outperforms training-free baselines, including CARLA TM and MADA, on core closed-loop metrics. For reproducibility, code is available at https://github.com/pku-smart-city/source_code/tree/main/RIA.