mlsanityData to 7 Oct 2026

Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline

arXiv 2206.08129 ↗First posted 16 Jun 2022cs.CV cs.AI cs.ROcited by 247

On the map: End-to-end autonomous driving → End-to-end system architecture

Abstract (arXiv)

Current end-to-end autonomous driving methods either run a controller based on a planned trajectory or perform control prediction directly, which have spanned two separately studied lines of research. Seeing their potential mutual benefits to each other, this paper takes the initiative to explore the combination of these two well-developed worlds. Specifically, our integrated approach has two branches for trajectory planning and direct control, respectively. The trajectory branch predicts the future trajectory, while the control branch involves a novel multi-step prediction scheme such that the relationship between current actions and future states can be reasoned. The two branches are connected so that the control branch receives corresponding guidance from the trajectory branch at each time step. The outputs from two branches are then fused to achieve complementary advantages. Our results are evaluated in the closed-loop urban driving setting with challenging scenarios using the CARLA simulator. Even with a monocular camera input, the proposed approach ranks first on the official CARLA Leaderboard, outperforming other complex candidates with multiple sensors or fusion mechanisms by a large margin. The source code is publicly available at https://github.com/OpenPerceptionX/TCP

Key facts. Cited by 247 papers in the map (data through 7 Oct 2026). Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving. Builds on 45 of its 72 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

45 of 72 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Driving19 Apr
  2. 2021End-to-end Interpretable Neural Motion Planner17 Jan

Other references · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  7. 2016End to End Learning for Self-Driving Carscited by 1,000
  8. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  9. 2019Learning by Cheatingcited by 361
  10. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 343
  11. 2018Learning to Drive in a Daycited by 266
  12. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
31 more
  1. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  2. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  3. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 191
  4. 2020Eigen-CAM: Class Activation Map using Principal Componentscited by 188
  5. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  6. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 150
  7. 2016End-to-end Learning of Driving Models from Large-scale Video Datasetscited by 147
  8. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  9. 2022PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmarkcited by 140
  10. 2022Learning from All Vehiclescited by 135
  11. 2018Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifierscited by 126
  12. 2021Learning to drive from a world on railscited by 114
  13. 2019Multimodal End-to-End Autonomous Drivingcited by 110
  14. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 96
  15. 2019Urban Driving with Conditional Imitation Learningcited by 84
  16. 2021YOLOP: You Only Look Once for Panoptic Driving Perceptioncited by 82
  17. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  18. 2018Deep Imitative Models for Flexible Inference, Planning, and Controlcited by 75
  19. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 59
  20. 2021OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Drivingcited by 57
  21. 2021Multi-task Learning with Attention for End-to-end Autonomous Drivingcited by 49
  22. 2019MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learningcited by 45
  23. 2018End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perceptioncited by 27
  24. 2021Learning by Watchingcited by 22
  25. 2018Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networkscited by 19
  26. 2022CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-based Autonomous Urban Drivingcited by 16
  27. 2019Deep Local Trajectory Replanning and Control for Robot Navigationcited by 15
  28. 2020The NVIDIA PilotNet Experimentscited by 15
  29. 2018Rethinking Self-driving: Multi-task Knowledge for Better Generalization and Accident Explanation Abilitycited by 12
  30. 2021Enhancing Object Detection for Autonomous Driving by Optimizing Anchor Generation and Addressing Class Imbalancecited by 6
  31. 2022Multi-Task Conditional Imitation Learning for Autonomous Navigation at Crowded Intersectionscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 36; 2024: 44; 2025: 86; 2026: 78.

247 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  4. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  5. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  6. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  7. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  8. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  9. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  10. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  11. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
  12. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
38 more
  1. 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
  2. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  3. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  4. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  5. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Modelcited by 145
  6. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 142
  7. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 142
  8. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  9. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 125
  10. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  11. 2023ADriver-I: A General World Model for Autonomous Drivingcited by 102
  12. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  13. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  14. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  15. 2023Hidden Biases of End-to-End Driving Modelscited by 95
  16. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 93
  17. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
  18. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  19. 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervisioncited by 85
  20. 2025DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Drivingcited by 85
  21. 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
  22. 2024DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Drivingcited by 67
  23. 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 66
  24. 2024CarLLaVA: Vision language models for camera-only closed-loop drivingcited by 57
  25. 2024Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)cited by 51
  26. 2024HIMap: HybrId Representation Learning for End-to-end Vectorized HD Map Constructioncited by 48
  27. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 46
  28. 2024Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Drivingcited by 46
  29. 2025Distilling Multi-modal Large Language Models for Autonomous Drivingcited by 46
  30. 2025Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)cited by 45
  31. 2023Pyramid Diffusion for Fine 3D Large Scene Generationcited by 44
  32. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  33. 2025ReSim: Reliable World Simulation for Autonomous Drivingcited by 42
  34. 2024DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Drivingcited by 41
  35. 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 41
  36. 2025iPad: Iterative Proposal-centric End-to-End Autonomous Drivingcited by 41
  37. 2024Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Drivingcited by 40
  38. 2025Hydra-NeXt: Robust Closed-Loop Driving with Open-Loop Trainingcited by 39

50 most cited of 247 All 247 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms
  2. 2026Addressing the Waypoint-Action Gap in End-to-End Autonomous Driving via Vehicle Motion Models
  3. 2026Guiding End-to-End Driving Models with Endpoint-Constrained Trajectory Optimization
  4. 2023End-to-end Autonomous Driving using Deep Learning: A Systematic Review
  5. 2026Post-Training in End-to-End Autonomous Driving
  6. 2025Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution
  7. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectives
  8. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platform