mlsanityData to 7 Oct 2026

DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving

arXiv 2503.07656 ↗First posted 7 Mar 2025cs.LG cs.CV cs.ROcited by 125

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

End-to-end autonomous driving (E2E-AD) has emerged as a trend in the field of autonomous driving, promising a data-driven, scalable approach to system design. However, existing E2E-AD methods usually adopt the sequential paradigm of perception-prediction-planning, which leads to cumulative errors and training instability. The manual ordering of tasks also limits the system`s ability to leverage synergies between tasks (for example, planning-aware perception and game-theoretic interactive prediction and planning). Moreover, the dense BEV representation adopted by existing methods brings computational challenges for long-range perception and long-term temporal fusion. To address these challenges, we present DriveTransformer, a simplified E2E-AD framework for the ease of scaling up, characterized by three key features: Task Parallelism (All agent, map, and planning queries direct interact with each other at each block), Sparse Representation (Task queries direct interact with raw sensor features), and Streaming Processing (Task queries are stored and passed as history information). As a result, the new framework is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention, which significantly reduces the complexity of system and leads to better training stability. DriveTransformer achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.

Key facts. Cited by 125 papers in the map (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 58 of its 67 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

58 of 67 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving6 Jun
  2. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation30 May
  3. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Driving10 Apr
  4. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?5 Dec
  5. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving1 Aug
  6. 2023Hidden Biases of End-to-End Driving Models13 Jun
  7. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenes17 May
  8. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving10 May
  9. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning15 Jul
  10. 2022MMFN: Multi-Modal-Fusion-Net for End-to-End Driving1 Jul
  11. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline16 Jun
  12. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving31 May
2 more
  1. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Driving19 Apr
  2. 2021End-to-end Interpretable Neural Motion Planner17 Jan

Other references · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  9. 2022Planning-oriented Autonomous Drivingcited by 1,068
  10. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  11. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  12. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
32 more
  1. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  2. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  3. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  4. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  5. 2023EVA-02: A Visual Representation for Neon Genesiscited by 377
  6. 2019Learning by Cheatingcited by 361
  7. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  8. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  9. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  10. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  11. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  12. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
  13. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  14. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  15. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  16. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  17. 2022PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmarkcited by 140
  18. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  19. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  20. 2022Learning from All Vehiclescited by 135
  21. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  22. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  23. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  24. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
  25. 2022Differentiable Raycasting for Self-supervised Occupancy Forecastingcited by 83
  26. 2023Far3D: Expanding the Horizon for Surround-view 3D Object Detectioncited by 70
  27. 2023LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Drivingcited by 57
  28. 2024Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)cited by 51
  29. 2020Learning to Evaluate Perception Models Using Planner-Centric Metricscited by 49
  30. 2024Leveraging Enhanced Queries of Point Sets for Vectorized Map Constructioncited by 46
  31. 2023RoboBEV: Towards Robust Bird's Eye View Perception under Corruptionscited by 42
  32. 2020IDE-Net: Interactive Driving Event and Pattern Extraction from Human Datacited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 57; 2026: 68.

125 papers.

Sort:

By area:

  1. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  2. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
  3. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
  4. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 91
  5. 2025DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Drivingcited by 85
  6. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  7. 2025Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)cited by 45
  8. 2025ReSim: Reliable World Simulation for Autonomous Drivingcited by 42
  9. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  10. 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 39
  11. 2026Driving on Registerscited by 38
  12. 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decodercited by 35
38 more
  1. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 34
  2. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  3. 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
  4. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Drivingcited by 27
  5. 2026Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Drivingcited by 26
  6. 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
  7. 2025LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Drivingcited by 22
  8. 2026SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Drivingcited by 21
  9. 2025StyleDrive: Towards Driving-Style Aware Benchmarking of End-To-End Autonomous Drivingcited by 20
  10. 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Drivingcited by 20
  11. 2026DriveVA: Video Action Models are Zero-Shot Driverscited by 19
  12. 2025DSDrive: Distilling Large Language Model for Lightweight End-to-End Autonomous Driving with Unified Reasoning and Planningcited by 17
  13. 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Drivingcited by 17
  14. 2025BridgeDrive: Diffusion Bridge Policy for Closed-Loop Trajectory Planning in Autonomous Drivingcited by 17
  15. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 17
  16. 2026Devil is in Narrow Policy: Unleashing Exploration in Driving VLA Modelscited by 14
  17. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representationcited by 13
  18. 2025ResAD: Normalized Residual Trajectory Modeling for End-to-End Autonomous Drivingcited by 12
  19. 2025OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Criticcited by 11
  20. 2026DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Drivingcited by 11
  21. 2025DRoPE: Directional Rotary Position Embedding for Efficient Agent Interaction Modelingcited by 9
  22. 2025ETA: Efficiency through Thinking Ahead, A Dual Approach to Self-Driving with Large Modelscited by 9
  23. 2026DriveMamba: Task-Centric Scalable State Space Model for Efficient End-to-End Autonomous Drivingcited by 9
  24. 2026Unifying Language-Action Understanding and Generation for Autonomous Drivingcited by 9
  25. 2026AutoMoT: A Unified Vision-Language-Action Model with Asynchronous Mixture-of-Transformers for End-to-End Autonomous Drivingcited by 9
  26. 2025RoCA: Robust Cross-Domain End-to-End Autonomous Drivingcited by 8
  27. 2025PlanT 2.0: Exposing Biases and Structural Flaws in Closed-Loop Drivingcited by 8
  28. 2025TrajDiff: End-to-end Autonomous Driving without Perception Annotationcited by 8
  29. 2025TakeAD: Preference-based Post-optimization for End-to-end Autonomous Driving with Expert Takeover Datacited by 8
  30. 2026Learning from Mistakes: Post-Training for Driving VLA with Takeover Datacited by 8
  31. 2025CoReVLA: A Dual-Stage End-to-End Autonomous Driving Framework for Long-Tail Scenarios via Collect-and-Refinecited by 7
  32. 2025FutureX: Enhance End-to-End Autonomous Driving via Latent Chain-of-Thought World Modelcited by 7
  33. 2026SafeDrive: Fine-Grained Safety Reasoning for End-to-End Driving in a Sparse Worldcited by 7
  34. 2025ALN-P3: Unified Language Alignment for Perception, Prediction, and Planning in Autonomous Drivingcited by 6
  35. 2026Generative Scenario Rollouts for End-to-End Autonomous Drivingcited by 6
  36. 2026Efficient-LVSM: Faster, Cheaper, and Better Large View Synthesis Model via Decoupled Co-Refinement Attentioncited by 6
  37. 2026GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Drivingcited by 6
  38. 2025Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectivescited by 5

50 most cited of 125 All 125 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform
  2. 2026StandardE2E: A Unified Framework for End-to-End Autonomous Driving Datasets
  3. 2026Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms
  4. 2025DiffE2E: Rethinking End-to-End Driving with a Hybrid Action Diffusion and Supervised Policy
  5. 2026End-to-End Learning vs. Modular Architectures: Comparative Insights into Autonomous Driving Systems
  6. 2026FROST-Drive: Scalable and Efficient End-to-End Driving with a Frozen Vision Encoder
  7. 2024Generalizing Motion Planners with Mixture of Experts for Autonomous Driving
  8. 2026What Matters for Scalable and Robust Learning in End-to-End Driving Planners?