mlsanityData to 7 Oct 2026

Transformer-Based Sensor Fusion for Autonomous Driving: A Survey

arXiv 2302.11481 ↗First posted 22 Feb 2023cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Sensor fusion is an essential topic in many perception systems, such as autonomous driving and robotics. Transformers-based detection head and CNN-based feature encoder to extract features from raw sensor-data has emerged as one of the best performing sensor-fusion 3D-detection-framework, according to the dataset leaderboards. In this work we provide an in-depth literature survey of transformer based 3D-object detection task in the recent past, primarily focusing on the sensor fusion. We also briefly go through the Vision transformers (ViT) basics, so that readers can easily follow through the paper. Moreover, we also briefly go through few of the non-transformer based less-dominant methods for sensor fusion for autonomous driving. In conclusion we summarize with sensor-fusion trends to follow and provoke future research. More updated summary can be found at: https://github.com/ApoorvRoboticist/Transformers-Sensor-Fusion

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 26 of its 29 references that are in the map. First posted on arXiv 22 Feb 2023.

Builds on

26 of 29 references.

References · 26

  1. 2020End-to-End Object Detection with Transformerscited by 7,384
  2. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  6. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  7. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  8. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  9. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  10. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  11. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  12. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
14 more
  1. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  2. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  3. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  4. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  5. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  6. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  7. 2020CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detectioncited by 147
  8. 2021Efficient DETR: Improving End-to-End Object Detector with Dense Priorcited by 128
  9. 2022DeepInteraction: 3D Object Detection via Modality Interactioncited by 125
  10. 2022AutoAlign: Pixel-Instance Feature Aggregation for Multi-Modal 3D Object Detectioncited by 70
  11. 20214D-Net for Learned Multi-Modal Alignmentcited by 44
  12. 2023Vision-RADAR fusion for Robotics BEV Detections: A Surveycited by 14
  13. 2023Surround-View Vision-based 3D Detection for Autonomous Driving: A Surveycited by 14
  14. 20233M3D: Multi-view, Multi-path, Multi-representation for 3D Object Detectioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 4; 2025: 3; 2026: 1.

14 papers.

Sort:

By area:

  1. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  2. 2023MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusioncited by 19
  3. 2023VT-Former: An Exploratory Study on Vehicle Trajectory Prediction for Highway Surveillance through Graph Isomorphism and Transformercited by 10
  4. 2025Hyperdimensional Uncertainty Quantification for Multimodal Uncertainty Fusion in Autonomous Vehicles Perceptioncited by 6
  5. 2024XLM for Autonomous Driving Systems: A Comprehensive Reviewcited by 5
  6. 2023Trajectory-Prediction with Vision: A Surveycited by 4
  7. 2023So you think you can track?cited by 4
  8. 2024Hierarchical and Decoupled BEV Perception Learning Framework for Autonomous Drivingcited by 2
  9. 2024METDrive: Multi-modal End-to-end Autonomous Driving with Temporal Guidancecited by 2
  10. 2023Training Strategies for Vision Transformers for Object Detectioncited by 1
  11. 2023Lidar based 3D Tracking and State Estimation of Dynamic Objectscited by 0
  12. 2025HiLO: High-Level Object Fusion for Autonomous Driving using Transformerscited by 0
2 more
  1. 2025Deep Learning Perspective of Scene Understanding in Autonomous Robotscited by 0
  2. 2026Co-Fusion4D: Spatio-temporal Collaborative Fusion for Robust 3D Object Detectioncited by 0