mlsanityData to 14 Oct 2026

PlanT: Explainable Planning Transformers via Object-Level Representations

arXiv 2210.14222 ↗First posted 25 Oct 2022cs.RO cs.AI cs.CVcited by 105

Not on the map. Why?

Cited by 71 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Planning an optimal route in a complex environment requires efficient reasoning about the surrounding scene. While human drivers prioritize important objects and ignore details not relevant to the decision, learning-based planners typically extract features from dense, high-dimensional grid representations containing all vehicle and road context information. In this paper, we propose PlanT, a novel approach for planning in the context of self-driving that uses a standard transformer architecture. PlanT is based on imitation learning with a compact object-level input representation. On the Longest6 benchmark for CARLA, PlanT outperforms all prior methods (matching the driving score of the expert) while being 5.3x faster than equivalent pixel-based planning baselines during inference. Combining PlanT with an off-the-shelf perception module provides a sensor-based driving system that is more than 10 points better in terms of driving score than the existing state of the art. Furthermore, we propose an evaluation protocol to quantify the ability of planners to identify relevant objects, providing insights regarding their decision-making. Our results indicate that PlanT can focus on the most relevant object in the scene, even when this object is geometrically distant.

Key facts. Cited by 105 papers in the map (data through 14 Oct 2026). Builds on 61 of its 75 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

61 of 75 references.

References · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  6. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  8. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  9. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  10. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  11. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  12. 2019Objects as Pointscited by 1,175
38 more
  1. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  2. 2022A Generalist Agentcited by 808
  3. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  4. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  5. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  6. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  7. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  8. 2019Learning by Cheatingcited by 383
  9. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 377
  10. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 346
  11. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  12. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
  13. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  14. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 254
  15. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  16. 2021End-to-end Interpretable Neural Motion Plannercited by 242
  17. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
  18. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  19. 2021Multimodal Motion Prediction with Stacked Transformerscited by 208
  20. 2022Online Decision Transformercited by 196
  21. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 194
  22. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  23. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  24. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 156
  25. 2020Perceive, Predict, and Plan: Safe Motion Planning Through Interpretable Semantic Representationscited by 153
  26. 2022Learning from All Vehiclescited by 135
  27. 2022BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigscited by 123
  28. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  29. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  30. 2021Translating Images into Mapscited by 115
  31. 2017Interpretable Learning for Self-Driving Cars by Visualizing Causal Attentioncited by 107
  32. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 106
  33. 2018Baidu Apollo EM Motion Plannercited by 100
  34. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  35. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradientscited by 96
  36. 2018Driving Policy Transfer via Modularity and Abstractioncited by 92
  37. 2018Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Drivingcited by 90
  38. 2018Visual Representations for Semantic Target Driven Navigationcited by 89

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 23; 2024: 32; 2025: 27; 2026: 23.

105 papers.

Sort:

By area:

  1. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  2. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  4. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  5. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  6. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  7. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 242
  8. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 233
  9. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 232
  10. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 169
  11. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 166
  12. 2025Pseudo-Simulation for Autonomous Drivingcited by 150
38 more
  1. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 144
  2. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  3. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 103
  4. 2023Hidden Biases of End-to-End Driving Modelscited by 96
  5. 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Modelcited by 92
  6. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  7. 2025Robust Autonomy Emerges from Self-Playcited by 67
  8. 2024CarLLaVA: Vision language models for camera-only closed-loop drivingcited by 57
  9. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Followingcited by 42
  10. 2024UnO: Unsupervised Occupancy Fields for Perception and Forecastingcited by 41
  11. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  12. 2024Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?cited by 38
  13. 2024Asynchronous Large Language Model Enhanced Planner for Autonomous Drivingcited by 38
  14. 2025CaRL: Learning Scalable Planning Policies with Simple Rewardscited by 35
  15. 2025CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-scale Reinforcement Learning in Autonomous Drivingcited by 34
  16. 2023PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Drivingcited by 33
  17. 2024Hidden Biases of End-to-End Driving Datasetscited by 30
  18. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  19. 2024SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Trafficcited by 27
  20. 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
  21. 2023Large Trajectory Models are Scalable Motion Predictors and Plannerscited by 23
  22. 2024Generalizing Motion Planners with Mixture of Experts for Autonomous Drivingcited by 22
  23. 2024Hybrid-Prediction Integrated Planning for Autonomous Drivingcited by 18
  24. 2023Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Drivingcited by 17
  25. 2024ActiveAD: Planning-Oriented Active Learning for End-to-End Autonomous Drivingcited by 16
  26. 2025VERDI: VLM-Embedded Reasoning for Autonomous Drivingcited by 16
  27. 2025Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorercited by 16
  28. 2024FREA: Feasibility-Guided Generation of Safety-Critical Scenarios with Reasonable Adversarialitycited by 13
  29. 2023Action-slot: Visual Action-centric Representations for Multi-label Atomic Activity Recognition in Traffic Scenescited by 12
  30. 2023RiskBench: A Scenario-based Benchmark for Risk Identificationcited by 12
  31. 2025Unraveling the Effects of Synthetic Data on End-to-End Autonomous Drivingcited by 12
  32. 2023On Offline Evaluation of 3D Object Detection for Autonomous Drivingcited by 11
  33. 2024QuAD: Query-based Interpretable Neural Motion Planning for Autonomous Drivingcited by 11
  34. 2024Instruct Large Language Models to Drive like Humanscited by 11
  35. 2025DriveQA: Passing the Driving Knowledge Testcited by 10
  36. 2025DiffE2E: Rethinking End-to-End Driving with a Hybrid Action Diffusion and Supervised Policycited by 9
  37. 2024CarFormer: Self-Driving with Learned Object-Centric Representationscited by 8
  38. 2024Learning Multiple Probabilistic Decisions from Latent World Model in Autonomous Drivingcited by 8

50 most cited of 105 All 105 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving
  2. 2026CarPLAN: Context-Adaptive and Robust Planning with Dynamic Scene Awareness for Autonomous Driving
  3. 2022Hierarchical Model-Based Imitation Learning for Planning in Autonomous Driving
  4. 2025LeAD: The LLM Enhanced Planning System Converged with End-to-end Autonomous Driving
  5. 2022Model-Based Imitation Learning for Urban Driving
  6. 2021Exploring Imitation Learning for Autonomous Driving with Feedback Synthesizer and Differentiable Rasterization
  7. 2024Hybrid Imitation-Learning Motion Planner for Urban Driving
  8. 2026Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms