mlsanityData to 14 Oct 2026

Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer

arXiv 2207.14024 ↗First posted 28 Jul 2022cs.CV cs.AI cs.LGcited by 212

Not on the map. Why?

Cited by 142 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large-scale deployment of autonomous vehicles has been continually delayed due to safety concerns. On the one hand, comprehensive scene understanding is indispensable, a lack of which would result in vulnerability to rare but complex traffic situations, such as the sudden emergence of unknown objects. However, reasoning from a global context requires access to sensors of multiple types and adequate fusion of multi-modal sensor signals, which is difficult to achieve. On the other hand, the lack of interpretability in learning models also hampers the safety with unverifiable failure causes. In this paper, we propose a safety-enhanced autonomous driving framework, named Interpretable Sensor Fusion Transformer(InterFuser), to fully process and fuse information from multi-modal multi-view sensors for achieving comprehensive scene understanding and adversarial event detection. Besides, intermediate interpretable features are generated from our framework, which provide more semantics and are exploited to better constrain actions to be within the safe sets. We conducted extensive experiments on CARLA benchmarks, where our model outperforms prior methods, ranking the first on the public CARLA Leaderboard. Our code will be made available at https://github.com/opendilab/InterFuser

Key facts. Cited by 212 papers in the map (data through 14 Oct 2026). Builds on 45 of its 60 references that are in the map. First posted on arXiv 28 Jul 2022.

Builds on

45 of 60 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2016Layer Normalizationcited by 5,669
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  10. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  11. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  12. 2019Objects as Pointscited by 1,175
33 more
  1. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  2. 2019Learning by Cheatingcited by 383
  3. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 377
  4. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  5. 2019Exploring the Limitations of Behavior Cloning for Autonomous Drivingcited by 346
  6. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 341
  7. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  8. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 248
  9. 2021End-to-end Interpretable Neural Motion Plannercited by 242
  10. 2019Algorithms for Verifying Deep Neural Networkscited by 221
  11. 2016Learning to Navigate in Complex Environmentscited by 198
  12. 2021End-to-End Urban Driving by Imitating a Reinforcement Learning Coachcited by 194
  13. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 179
  14. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  15. 2019Bottom-up Object Detection by Grouping Extreme and Center Pointscited by 156
  16. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 149
  17. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  18. 2022Learning from All Vehiclescited by 135
  19. 2021Learning to drive from a world on railscited by 114
  20. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 110
  21. 2021TransformerFusion: Monocular RGB Scene Reconstruction using Transformerscited by 106
  22. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 103
  23. 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
  24. 2021GRI: General Reinforced Imitation and its Application to Vision-Based Autonomous Drivingcited by 60
  25. 2018Probabilistic Prediction of Vehicle Semantic Intention and Motioncited by 31
  26. 2018Probabilistically Safe Robot Planning with Confidence-Based Human Predictionscited by 28
  27. 2020On Infusing Reachability-Based Safety Assurance within Planning Frameworks for Human-Robot Vehicle Interactionscited by 25
  28. 2022Transferable and Adaptable Driving Behavior Predictioncited by 20
  29. 2020Socially-Compatible Behavior Design of Autonomous Vehicles with Verification on Real Human Datacited by 18
  30. 2021Blending Anti-Aliasing into Vision Transformercited by 16
  31. 2021The Reasonable Crowd: Towards evidence-based and interpretable models of driving behaviorcited by 12
  32. 2021Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Predictioncited by 10
  33. 2021Online Adaptation of Neural Network Models by Modified Extended Kalman Filter for Customizable and Transferable Driving Behavior Predictioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 40; 2024: 62; 2025: 62; 2026: 44.

212 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,087
  2. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  3. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  4. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  5. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  6. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  7. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  8. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  9. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 194
  10. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 169
  11. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 166
  12. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 150
38 more
  1. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 144
  2. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  3. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 103
  4. 2023Hidden Biases of End-to-End Driving Modelscited by 96
  5. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 94
  6. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 93
  7. 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Modelcited by 92
  8. 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervisioncited by 86
  9. 2024PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Drivingcited by 84
  10. 2023GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformerscited by 79
  11. 2025A Survey on Vision-Language-Action Models for Autonomous Drivingcited by 72
  12. 2024DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Drivingcited by 67
  13. 2024CarLLaVA: Vision language models for camera-only closed-loop drivingcited by 57
  14. 2024Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Reviewcited by 53
  15. 2025Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)cited by 47
  16. 2024DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Drivingcited by 43
  17. 2024End-to-End Autonomous Driving through V2X Cooperationcited by 42
  18. 2024Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Drivingcited by 41
  19. 2022X-Align: Cross-Modal Cross-View Alignment for Bird's-Eye-View Segmentationcited by 38
  20. 2024Can Vehicle Motion Planning Generalize to Realistic Long-tail Scenarios?cited by 38
  21. 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Drivingcited by 38
  22. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 36
  23. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priorscited by 36
  24. 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decodercited by 35
  25. 2024SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Predictioncited by 34
  26. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  27. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  28. 2024Hidden Biases of End-to-End Driving Datasetscited by 30
  29. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  30. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  31. 2024End-to-End Autonomous Driving without Costly Modularization and 3D Manual Annotationcited by 28
  32. 2023Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modelingcited by 26
  33. 2025BEVDriver: Leveraging BEV Maps in LLMs for Robust Closed-Loop Drivingcited by 26
  34. 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
  35. 2025LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Drivingcited by 23
  36. 2024Mitigating Covariate Shift in Imitation Learning for Autonomous Vehicles Using Latent Space Generative World Modelscited by 22
  37. 2024BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latentscited by 21
  38. 2025SOLVE: Synergy of Language-Vision and End-to-End Networks for Autonomous Drivingcited by 21

50 most cited of 212 All 212 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025INSIGHT: Enhancing Autonomous Driving Safety through Vision-Language Models on Context-Aware Hazard Detection and Edge Case Evaluation
  2. 2025Integrating Multi-Modal Sensors: A Review of Fusion Techniques for Intelligent Vehicles
  3. 2026UniDrive: A Unified Vision-Language and Grounding Framework for Interpretable Risk Understanding in Autonomous Driving
  4. 2023LeTFuser: Light-weight End-to-end Transformer-Based Sensor Fusion for Autonomous Driving with Multi-Task Learning
  5. 2025Resilient Sensor Fusion under Adverse Sensor Failures via Multi-Modal Expert Fusion
  6. 2024DRIVE: Dependable Robust Interpretable Visionary Ensemble Framework in Autonomous Driving
  7. 2026Driving the Wrong Way: Leveraging Interpretability in End2End Autonomous Driving Models
  8. 2025Multimodal Large Language Models for Enhanced Traffic Safety: A Comprehensive Review and Future Trends