mlsanityData to 7 Oct 2026

Learning to drive from a world on rails

arXiv 2105.00636 ↗First posted 3 May 2021cs.RO cs.CV cs.LGcited by 114

On the map: End-to-end autonomous driving → Generative simulation for driving

Abstract (arXiv)

We learn an interactive vision-based driving policy from pre-recorded driving logs via a model-based approach. A forward model of the world supervises a driving policy that predicts the outcome of any potential driving trajectory. To support learning from pre-recorded logs, we assume that the world is on rails, meaning neither the agent nor its actions influence the environment. This assumption greatly simplifies the learning problem, factorizing the dynamics into a nonreactive world model and a low-dimensional and compact forward model of the ego-vehicle. Our approach computes action-values for each training trajectory using a tabular dynamic-programming evaluation of the Bellman equations; these action-values in turn supervise the final vision-based driving policy. Despite the world-on-rails assumption, the final driving policy acts well in a dynamic and reactive world. At the time of writing, our method ranks first on the CARLA leaderboard, attaining a 25% higher driving score while using 40 times less data. Our method is also an order of magnitude more sample-efficient than state-of-the-art model-free reinforcement learning techniques on navigational tasks in the ProcGen benchmark.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. Builds on 36 of its 47 references that are in the map. First posted on arXiv 3 May 2021.

Builds on

36 of 47 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  9. 2016End to End Learning for Self-Driving Carscited by 1,000
  10. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  11. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  12. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
24 more
  1. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  2. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  3. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  4. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  5. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  6. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  7. 2019Learning by Cheatingcited by 361
  8. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  9. 2021End-to-end Interpretable Neural Motion Plannercited by 240
  10. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
  11. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  12. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  13. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
  14. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 152
  15. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  16. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 149
  17. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  18. 2020Phasic Policy Gradientcited by 121
  19. 2017Value Prediction Networkcited by 117
  20. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 96
  21. 2018Driving Policy Transfer via Modularity and Abstractioncited by 91
  22. 2018Conditional Affordance Learning for Driving in Urban Environmentscited by 70
  23. 2018Discovering and Removing Exogenous State Variables and Rewards for Reinforcement Learningcited by 24
  24. 2019Learning Compact Models for Planning with Exogenous Processescited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 22; 2023: 28; 2024: 20; 2025: 30; 2026: 11.

114 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  4. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  5. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  6. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  7. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  8. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  9. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  10. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  11. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  12. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
38 more
  1. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  2. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  3. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  4. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  5. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 145
  6. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 142
  7. 2025Diffusion-Based Planning for Autonomous Driving with Flexible Guidancecited by 138
  8. 2022Learning from All Vehiclescited by 135
  9. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  10. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  11. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 95
  12. 2022COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehiclescited by 95
  13. 2023Hidden Biases of End-to-End Driving Modelscited by 95
  14. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
  15. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  16. 2023The Waymo Open Sim Agents Challengecited by 86
  17. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  18. 2025RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learningcited by 76
  19. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 59
  20. 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 52
  21. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  22. 2025ReSim: Reliable World Simulation for Autonomous Drivingcited by 42
  23. 2024Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Drivingcited by 40
  24. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  25. 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decodercited by 35
  26. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  27. 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
  28. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  29. 2022Addressing Optimism Bias in Sequence Modeling for Reinforcement Learningcited by 22
  30. 2025Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modelingcited by 22
  31. 2025LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Drivingcited by 22
  32. 2023IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracingcited by 20
  33. 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
  34. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  35. 2022Driving in Real Life with Inverse Reinforcement Learningcited by 18
  36. 2022MMFN: Multi-Modal-Fusion-Net for End-to-End Drivingcited by 17
  37. 2023Coaching a Teachable Studentcited by 17
  38. 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Drivingcited by 17

50 most cited of 114 All 114 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving
  2. 2024Imagine-2-Drive: Leveraging High-Fidelity World Models via Multi-Modal Diffusion Policies
  3. 2026Dreamer-SAC: Off-Policy Learning in Latent World Models for Sample-Efficient Autonomous Driving
  4. 2026DriveZero: End-to-End Driving Beyond Human Demonstrations
  5. 2026TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations
  6. 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Models
  7. 2024From Imitation to Exploration: End-to-end Autonomous Driving based on World Model
  8. 2026World Engine: Towards the Era of Post-Training for Autonomous Driving