mlsanityData to 7 Oct 2026

DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Driving

arXiv 2308.00398 ↗First posted 1 Aug 2023cs.RO cs.CVcited by 142

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

End-to-end autonomous driving aims to build a fully differentiable system that takes raw sensor data as inputs and directly outputs the planned trajectory or control signals of the ego vehicle. State-of-the-art methods usually follow the `Teacher-Student' paradigm. The Teacher model uses privileged information (ground-truth states of surrounding agents and map elements) to learn the driving strategy. The student model only has access to raw sensor data and conducts behavior cloning on the data collected by the teacher model. By eliminating the noise of the perception part during planning learning, state-of-the-art works could achieve better performance with significantly less data compared to those coupled ones. However, under the current Teacher-Student paradigm, the student model still needs to learn a planning head from scratch, which could be challenging due to the redundant and noisy nature of raw sensor inputs and the casual confusion issue of behavior cloning. In this work, we aim to explore the possibility of directly adopting the strong teacher model to conduct planning while letting the student model focus more on the perception part. We find that even equipped with a SOTA perception model, directly letting the student model learn the required inputs of the teacher model leads to poor driving performance, which comes from the large distribution gap between predicted privileged inputs and the ground-truth. To this end, we propose DriveAdapter, which employs adapters with the feature alignment objective function between the student (perception) and teacher (planning) modules. Additionally, since the pure learning-based teacher model itself is imperfect and occasionally breaks safety rules, we propose a method of action-guided feature learning with a mask for those imperfect teacher features to further inject the priors of hand-crafted rules into the learning process.

Key facts. Cited by 142 papers in the map (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 44 of its 58 references that are in the map. First posted on arXiv 1 Aug 2023.

Builds on

44 of 58 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023End-to-end Autonomous Driving: Challenges and Frontiers29 Jun
  2. 2023Hidden Biases of End-to-End Driving Models13 Jun
  3. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving10 May
  4. 2022MMFN: Multi-Modal-Fusion-Net for End-to-End Driving1 Jul
  5. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline16 Jun
  6. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving31 May
  7. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Driving19 Apr

Other references · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2017Squeeze-and-Excitation Networkscited by 4,763
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  8. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  9. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  10. 2022Planning-oriented Autonomous Drivingcited by 1,068
  11. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  12. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
25 more
  1. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  2. 2018Path Aggregation Network for Instance Segmentationcited by 796
  3. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  4. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  5. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  6. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 429
  7. 2019Learning by Cheatingcited by 361
  8. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  9. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
  10. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  11. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  12. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  13. 2022Model-Based Imitation Learning for Urban Drivingcited by 149
  14. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  15. 2022Learning from All Vehiclescited by 135
  16. 2021Learning to drive from a world on railscited by 114
  17. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 109
  18. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  19. 2020Fighting Copycat Agents in Behavioral Cloning from Observation Historiescited by 70
  20. 2023Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modelingcited by 25
  21. 2020IDE-Net: Interactive Driving Event and Pattern Extraction from Human Datacited by 19
  22. 2023Coaching a Teachable Studentcited by 17
  23. 2022Fighting Fire with Fire: Avoiding DNN Shortcuts through Primingcited by 15
  24. 2022Resolving Copycat Problems in Visual Imitation Learning via Residual Action Predictioncited by 14
  25. 2022IBISCape: A Simulated Benchmark for multi-modal SLAM Systems Evaluation in Large-scale Dynamic Environmentscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 28; 2025: 52; 2026: 53.

142 papers.

Sort:

By area:

  1. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  2. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  3. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  4. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  5. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  6. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  7. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
  8. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  9. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  10. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  11. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  12. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 125
38 more
  1. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  2. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  3. 2025DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Drivingcited by 85
  4. 2025Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Modelscited by 60
  5. 2024Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)cited by 51
  6. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 46
  7. 2024Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Drivingcited by 46
  8. 2025Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)cited by 45
  9. 2025ReSim: Reliable World Simulation for Autonomous Drivingcited by 42
  10. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 41
  11. 2025iPad: Iterative Proposal-centric End-to-End Autonomous Drivingcited by 41
  12. 2025Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Trainingcited by 39
  13. 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decodercited by 35
  14. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 34
  15. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  16. 2024Hidden Biases of End-to-End Driving Datasetscited by 30
  17. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  18. 2024OccGen: Generative Multi-modal 3D Occupancy Prediction for Autonomous Drivingcited by 28
  19. 2025SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Drivingcited by 28
  20. 2024End-to-End Autonomous Driving without Costly Modularization and 3D Manual Annotationcited by 27
  21. 2025DiffAD: A Unified Diffusion Modeling Approach for Autonomous Drivingcited by 26
  22. 2025MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learningcited by 25
  23. 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 24
  24. 2023Open-sourced Data Ecosystem in Autonomous Driving: the Present and Futurecited by 22
  25. 2025ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Drivingcited by 22
  26. 2025GuideFlow: Constraint-Guided Flow Matching for Planning in End-to-End Autonomous Drivingcited by 21
  27. 2026SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Drivingcited by 21
  28. 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
  29. 2026UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Drivingcited by 20
  30. 2025Reinforced Refinement with Self-Aware Expansion for End-to-End Autonomous Drivingcited by 19
  31. 2024Hybrid-Prediction Integrated Planning for Autonomous Drivingcited by 18
  32. 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Drivingcited by 17
  33. 2026UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Drivingcited by 17
  34. 2024ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Drivingcited by 16
  35. 2025GaussianFusion: Gaussian-Based Multi-Sensor Fusion for End-to-End Autonomous Drivingcited by 16
  36. 2025Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorercited by 16
  37. 2025VERDI: VLM-Embedded Reasoning for Autonomous Drivingcited by 15
  38. 2023Leveraging Vision-Centric Multi-Modal Expertise for 3D Object Detectioncited by 13

50 most cited of 142 All 142 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms
  2. 2026DriveZero: End-to-End Driving Beyond Human Demonstrations
  3. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform
  4. 2025Perception in Plan: Coupled Perception and Planning for End-to-End Autonomous Driving
  5. 2024Guiding Attention in End-to-End Driving Models
  6. 2026TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrations
  7. 2025Learning to Drive from a World Model
  8. 2026Post-Training in End-to-End Autonomous Driving