# Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)

arXiv: https://arxiv.org/abs/2505.16394 · HTML: https://mlsanity.com/paper/2505.16394/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Placed in World models for autonomous driving, one of 11 directions in World models.
- Builds on 48 of its 72 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- World models / [World models for autonomous driving](https://mlsanity.com/wm/world-models-for-autonomous-driving/)
- End-to-end autonomous driving / [Imitation-learning end-to-end planners](https://mlsanity.com/driving/imitation-learning-end-to-end-planners/)
- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 30 of 48)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [VAD: Vectorized Scene Representation for Efficient Autonomous Driving](https://mlsanity.com/paper/2303.12077/) (2023)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving](https://mlsanity.com/paper/2406.03877/) (2024)
- [Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline](https://mlsanity.com/paper/2206.08129/) (2022)
- [GenAD: Generative End-to-End Autonomous Driving](https://mlsanity.com/paper/2402.11502/) (2024)
- [Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer](https://mlsanity.com/paper/2207.14024/) (2022)
- [End-to-End Urban Driving by Imitating a Reinforcement Learning Coach](https://mlsanity.com/paper/2108.08265/) (2021)
- [Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes](https://mlsanity.com/paper/2305.10430/) (2023)
- [Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving](https://mlsanity.com/paper/2305.06242/) (2023)
- [Enhancing End-to-End Autonomous Driving with Latent World Model](https://mlsanity.com/paper/2406.08481/) (2024)
- [End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordances](https://mlsanity.com/paper/1911.10868/) (2019)
- [DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving](https://mlsanity.com/paper/2308.00398/) (2023)
- [LLM4Drive: A Survey of Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.01043/) (2023)
- [DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving](https://mlsanity.com/paper/2503.07656/) (2025)
- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videos](https://mlsanity.com/paper/2401.12592/) (2024)
- [HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encoding](https://mlsanity.com/paper/2205.09753/) (2022)
- [Model-free Deep Reinforcement Learning for Urban Autonomous Driving](https://mlsanity.com/paper/1904.09503/) (2019)

## Built on by (most cited first, 30 of 47)

- [A Comprehensive Survey on World Models for Embodied AI](https://mlsanity.com/paper/2510.16732/) (2025)
- [ReSim: Reliable World Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.09981/) (2025)
- [MIND: Benchmarking Memory Consistency and Action Control in World Models](https://mlsanity.com/paper/2602.08025/) (2026)
- [Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future](https://mlsanity.com/paper/2512.16760/) (2025)
- [SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving](https://mlsanity.com/paper/2512.10719/) (2025)
- [MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning](https://mlsanity.com/paper/2512.13636/) (2025)
- [DriveVA: Video Action Models are Zero-Shot Drivers](https://mlsanity.com/paper/2604.04198/) (2026)
- [PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulation](https://mlsanity.com/paper/2601.07060/) (2026)
- [Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorer](https://mlsanity.com/paper/2510.24108/) (2025)
- [EponaV2: Driving World Model with Comprehensive Future Reasoning](https://mlsanity.com/paper/2605.14696/) (2026)
- [AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Models](https://mlsanity.com/paper/2511.20325/) (2025)
- [MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2512.04441/) (2025)
- [DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving](https://mlsanity.com/paper/2603.11041/) (2026)
- [DriveMamba: Task-Centric Scalable State Space Model for Efficient End-to-End Autonomous Driving](https://mlsanity.com/paper/2602.13301/) (2026)
- [AutoMoT: A Unified Vision-Language-Action Model with Asynchronous Mixture-of-Transformers for End-to-End Autonomous Driving](https://mlsanity.com/paper/2603.14851/) (2026)
- [SimWAM: A Simple World Action Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2608.07468/) (2026)
- [PlanT 2.0: Exposing Biases and Structural Flaws in Closed-Loop Driving](https://mlsanity.com/paper/2511.07292/) (2025)
- [Learning from Mistakes: Post-Training for Driving VLA with Takeover Data](https://mlsanity.com/paper/2603.14972/) (2026)
- [World Engine: Towards the Era of Post-Training for Autonomous Driving](https://mlsanity.com/paper/2606.19836/) (2026)
- [GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving](https://mlsanity.com/paper/2606.16274/) (2026)
- [HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving](https://mlsanity.com/paper/2604.03581/) (2026)
- [Large Language Model-assisted Autonomous Vehicle Recovery from Immobilization](https://mlsanity.com/paper/2510.26023/) (2025)
- [Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges](https://mlsanity.com/paper/2603.09086/) (2026)
- [Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbations](https://mlsanity.com/paper/2605.18059/) (2026)
- [Distill to Think, Foresee to Act: Cognitive-Physical Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2605.21139/) (2026)
- [DriveZero: End-to-End Driving Beyond Human Demonstrations](https://mlsanity.com/paper/2609.06055/) (2026)
- [Driving Beyond Privilege: Distilling Dense-Reward Knowledge into Sparse-Reward Policies](https://mlsanity.com/paper/2512.04279/) (2025)
- [CRAFT: Counterfactual-to-Interactive Reinforcement Fine-Tuning for Driving Policies](https://mlsanity.com/paper/2605.04470/) (2026)
- [CLEAR: Closed-Loop Reinforcement Learning at Scale for End-to-End Autonomous Driving](https://mlsanity.com/paper/2607.02841/) (2026)
- [PerlAD: Towards Enhanced Closed-loop End-to-end Autonomous Driving with Pseudo-simulation-based Reinforcement Learning](https://mlsanity.com/paper/2603.14908/) (2026)

## Abstract (arXiv)

Reinforcement Learning (RL) can mitigate the causal confusion and distribution shift inherent to imitation learning (IL). However, applying RL to end-to-end autonomous driving (E2E-AD) remains an open problem for its training difficulty, and IL is still the mainstream paradigm in both academia and industry. Recently Model-based Reinforcement Learning (MBRL) have demonstrated promising results in neural planning; however, these methods typically require privileged information as input rather than raw sensor data. We fill this gap by designing Raw2Drive, a dual-stream MBRL approach. Initially, we efficiently train an auxiliary privileged world model paired with a neural planner that uses privileged information as input. Subsequently, we introduce a raw sensor world model trained via our proposed Guidance Mechanism, which ensures consistency between the raw sensor world model and the privileged world model during rollouts. Finally, the raw sensor world model combines the prior knowledge embedded in the heads of the privileged world model to effectively guide the training of the raw sensor policy. Raw2Drive is so far the only RL based end-to-end method on CARLA Leaderboard 2.0, and Bench2Drive and it achieves state-of-the-art performance.