mlsanityData to 14 Oct 2026

BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation

arXiv 2205.13542 ↗First posted 26 May 2022cs.CVcited by 850

Not on the map. Why?

Cited by 95 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Multi-sensor fusion is essential for an accurate and reliable autonomous driving system. Recent approaches are based on point-level fusion: augmenting the LiDAR point cloud with camera features. However, the camera-to-LiDAR projection throws away the semantic density of camera features, hindering the effectiveness of such methods, especially for semantic-oriented tasks (such as 3D scene segmentation). In this paper, we break this deeply-rooted convention with BEVFusion, an efficient and generic multi-task multi-sensor fusion framework. It unifies multi-modal features in the shared bird's-eye view (BEV) representation space, which nicely preserves both geometric and semantic information. To achieve this, we diagnose and lift key efficiency bottlenecks in the view transformation with optimized BEV pooling, reducing latency by more than 40x. BEVFusion is fundamentally task-agnostic and seamlessly supports different 3D perception tasks with almost no architectural changes. It establishes the new state of the art on nuScenes, achieving 1.3% higher mAP and NDS on 3D object detection and 13.6% higher mIoU on BEV map segmentation, with 1.9x lower computation cost. Code to reproduce our results is available at https://github.com/mit-han-lab/bevfusion.

Key facts. Cited by 850 papers in the map (data through 14 Oct 2026). Builds on 68 of its 78 references that are in the map. First posted on arXiv 26 May 2022.

Builds on

68 of 78 references.

References · 68

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2017Mask R-CNNcited by 7,277
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  7. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  9. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  10. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  11. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  12. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
38 more
  1. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  2. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  3. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  4. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  5. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  6. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  7. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  8. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  9. 2017Focal Loss for Dense Object Detectioncited by 752
  10. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  11. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  12. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  13. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  14. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  15. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  16. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  17. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  18. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  19. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  20. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  21. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  22. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 368
  23. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  24. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  25. 2021HDMapNet: An Online HD Map Construction and Evaluation Frameworkcited by 281
  26. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  27. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  28. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  29. 2021PV-RCNN++: Point-Voxel Feature Set Abstraction With Local Vector Representation for 3D Object Detectioncited by 232
  30. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  31. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  32. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 202
  33. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  34. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  35. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 191
  36. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  37. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  38. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 174

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 46; 2023: 187; 2024: 247; 2025: 200; 2026: 170.

850 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,087
  2. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  3. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  4. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 485
  5. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  6. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 360
  7. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
  8. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  9. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  10. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 314
  11. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  12. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 261
38 more
  1. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
  2. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  3. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  4. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  5. 2023VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Trackingcited by 222
  6. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  7. 2023Scene as Occupancycited by 209
  8. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 193
  9. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 178
  10. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 169
  11. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
  12. 2022DETRs with Hybrid Matchingcited by 150
  13. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  14. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  15. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 144
  16. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  17. 2023LLM4Drive: A Survey of Large Language Models for Autonomous Drivingcited by 136
  18. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  19. 2022Vision-Centric BEV Perception: A Surveycited by 126
  20. 2022DeepInteraction: 3D Object Detection via Modality Interactioncited by 125
  21. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  22. 2023DSVT: Dynamic Sparse Voxel Transformer with Rotated Setscited by 122
  23. 2023Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Drivingcited by 114
  24. 2023Language Prompt for Autonomous Drivingcited by 111
  25. 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
  26. 2023FlashOcc: Fast and Memory-Efficient Occupancy Prediction via Channel-to-Height Plugincited by 103
  27. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 99
  28. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  29. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 96
  30. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 93
  31. 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 92
  32. 2024Editable Scene Simulation for Autonomous Driving via Collaborative LLM-Agentscited by 91
  33. 2024An Extensible Framework for Open Heterogeneous Collaborative Perceptioncited by 87
  34. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 86
  35. 2023SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detectioncited by 83
  36. 2023CRN: Camera Radar Net for Accurate, Robust, Efficient 3D Perceptioncited by 82
  37. 2024RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detectioncited by 82
  38. 2023Virtual Sparse Convolution for Multimodal 3D Object Detectioncited by 81

50 most cited of 850 All 850 (JSON)