mlsanityData to 14 Oct 2026

Learning from All Vehicles

arXiv 2203.11934 ↗First posted 22 Mar 2022cs.RO cs.CV cs.LGcited by 135

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

In this paper, we present a system to train driving policies from experiences collected not just from the ego-vehicle, but all vehicles that it observes. This system uses the behaviors of other agents to create more diverse driving scenarios without collecting additional data. The main difficulty in learning from other vehicles is that there is no sensor information. We use a set of supervisory tasks to learn an intermediate representation that is invariant to the viewpoint of the controlling vehicle. This not only provides a richer signal at training time but also allows more complex reasoning during inference. Learning how all vehicles drive helps predict their behavior at test time and can avoid collisions. We evaluate this system in closed-loop driving simulations. Our system outperforms all prior methods on the public CARLA Leaderboard by a wide margin, improving driving score by 25 and route completion rate by 24 points. Our method won the 2021 CARLA Autonomous Driving challenge. Code and data are available at https://github.com/dotchen/LAV.

Key facts. Cited by 135 papers in the map (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 42 of its 54 references that are in the map. First posted on arXiv 22 Mar 2022.

Builds on

42 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Learning by Watching10 Jun

Other references · 41

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  6. 2019Objects as Pointscited by 1,175
  7. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  8. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  9. 2017End-to-end Driving via Conditional Imitation Learningcited by 485
  10. 2020Tracking Objects as Pointscited by 483
  11. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  12. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
29 more
  1. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 397
  2. 2019Learning by Cheatingcited by 383
  3. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 377
  4. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  5. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  6. 2021HDMapNet: An Online HD Map Construction and Evaluation Frameworkcited by 281
  7. 2018Learning to Drive in a Daycited by 273
  8. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  9. 2021End-to-end Interpretable Neural Motion Plannercited by 242
  10. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  11. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
  12. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  13. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 212
  14. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 193
  15. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  16. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  17. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  18. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 149
  19. 2021Learning to drive from a world on railscited by 114
  20. 20183D-LaneNet: End-to-End 3D Multiple Lane Detectioncited by 107
  21. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  22. 2020Gen-LaneNet: A Generalized and Scalable Approach for 3D Lane Detectioncited by 75
  23. 2018Conditional Affordance Learning for Driving in Urban Environmentscited by 70
  24. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 60
  25. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  26. 2021PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulationcited by 29
  27. 2019Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localizationcited by 24
  28. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  29. 2020StrObe: Streaming Object Detection from LiDAR Packetscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 11; 2023: 40; 2024: 35; 2025: 32; 2026: 17.

135 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,087
  2. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  4. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  5. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
  6. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  7. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
  8. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  9. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 248
  10. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 242
  11. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  12. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 212
38 more
  1. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 169
  2. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 166
  3. 2022Model-Based Imitation Learning for Urban Drivingcited by 150
  4. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 150
  5. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 144
  6. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  7. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  8. 2022PlanT: Explainable Planning Transformers via Object-Level Representationscited by 105
  9. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 104
  10. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 103
  11. 2023Hidden Biases of End-to-End Driving Modelscited by 96
  12. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 96
  13. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 93
  14. 2024An Extensible Framework for Open Heterogeneous Collaborative Perceptioncited by 87
  15. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  16. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  17. 2025Robust Autonomy Emerges from Self-Playcited by 67
  18. 2024CarLLaVA: Vision language models for camera-only closed-loop drivingcited by 57
  19. 2025Distilling Multi-modal Large Language Models for Autonomous Drivingcited by 46
  20. 2025ReSim: Reliable World Simulation for Autonomous Drivingcited by 44
  21. 2023Towards Vehicle-to-everything Autonomous Driving: A Survey on Collaborative Perceptioncited by 38
  22. 2024Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?cited by 38
  23. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 35
  24. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  25. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  26. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  27. 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
  28. 2024GraphAD: Interaction Scene Graph for End-to-end Autonomous Drivingcited by 25
  29. 2025LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Drivingcited by 23
  30. 2024Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Modelscited by 22
  31. 2023Learning and Aggregating Lane Graphs for Urban Automated Drivingcited by 20
  32. 2024Towards Collaborative Autonomous Driving: Simulation Platform and End-to-End Systemcited by 20
  33. 2024Is a 3D-Tokenized LLM the Key to Reliable Autonomous Driving?cited by 20
  34. 2023Coaching a Teachable Studentcited by 19
  35. 2025BridgeDrive: Diffusion Bridge Policy for Closed-Loop Trajectory Planning in Autonomous Drivingcited by 18
  36. 2026Fail2Drive: Benchmarking Closed-Loop Driving Generalizationcited by 17
  37. 2024ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Drivingcited by 16
  38. 2025VERDI: VLM-Embedded Reasoning for Autonomous Drivingcited by 16

50 most cited of 135 All 135 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning to Drive from a World Model
  2. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradients
  3. 2026Imitation Learning for Autonomous Driving in CARLA
  4. 2023End-to-end Autonomous Driving using Deep Learning: A Systematic Review
  5. 2026Post-Training in End-to-End Autonomous Driving
  6. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coach
  7. 2021Towards Learning Generalizable Driving Policies from Restricted Latent Representations
  8. 2026DriveZero: End-to-End Driving Beyond Human Demonstrations