mlsanityData to 7 Oct 2026

BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers

arXiv 2203.17270 ↗First posted 31 Mar 2022cs.CVcited by 1,213

On the map: World models → World models for autonomous driving

Abstract (arXiv)

3D visual perception tasks, including 3D detection and map segmentation based on multi-camera images, are essential for autonomous driving systems. In this work, we present a new framework termed BEVFormer, which learns unified BEV representations with spatiotemporal transformers to support multiple autonomous driving perception tasks. In a nutshell, BEVFormer exploits both spatial and temporal information by interacting with spatial and temporal space through predefined grid-shaped BEV queries. To aggregate spatial information, we design spatial cross-attention that each BEV query extracts the spatial features from the regions of interest across camera views. For temporal information, we propose temporal self-attention to recurrently fuse the history BEV information. Our approach achieves the new state-of-the-art 56.9% in terms of NDS metric on the nuScenes texttttest set, which is 9.0 points higher than previous best arts and on par with the performance of LiDAR-based baselines. We further show that BEVFormer remarkably improves the accuracy of velocity estimation and recall of objects under low visibility conditions. The code is available at urlhttps://github.com/zhiqi-li/BEVFormer.

Key facts. Cited by 1,213 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 50 of its 57 references that are in the map. First posted on arXiv 31 Mar 2022.

Builds on

50 of 57 references.

References · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  7. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  9. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  10. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  11. 2017Deformable Convolutional Networkscited by 1,481
  12. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
38 more
  1. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  2. 2019Objects as Pointscited by 1,172
  3. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  4. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  5. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  6. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  7. 2017Convolutional Sequence to Sequence Learningcited by 550
  8. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  9. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  10. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  11. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
  12. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  13. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  14. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  15. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  16. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  17. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  18. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 218
  19. 2019Disentangling Monocular 3D Object Detectioncited by 215
  20. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  21. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  22. 2016Deep Feature Flow for Video Recognitioncited by 186
  23. 2021NEAT: Neural Attention Fields for End-to-End Autonomous Drivingcited by 174
  24. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  25. 2019An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detectioncited by 163
  26. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  27. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  28. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  29. 2021Translating Images into Mapscited by 115
  30. 2021Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformerscited by 109
  31. 2021Structured Bird's-Eye-View Traffic Scene Understanding from Onboard Imagescited by 98
  32. 2020SSN: Shape Signature Networks for Multi-class Object Detection from Point Cloudscited by 74
  33. 2020A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird's Eye Viewcited by 71
  34. 20223D Object Detection from Images for Autonomous Driving: A Surveycited by 66
  35. 2020FISHING Net: Future Inference of Semantic Heatmaps In Gridscited by 55
  36. 2016Object Detection from Video Tubelets with Convolutional Neural Networkscited by 40
  37. 2020BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloudcited by 38
  38. 2018The Right (Angled) Perspective: Improving the Understanding of Road Scenes Using Boosted Inverse Perspective Mappingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 78; 2023: 282; 2024: 417; 2025: 282; 2026: 154.

1,213 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  3. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  4. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  5. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  6. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  7. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  8. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  9. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  10. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  11. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  12. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 359
38 more
  1. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  2. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  3. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  4. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  5. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  6. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  7. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  8. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  9. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  10. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  11. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  12. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  13. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  14. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  15. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 287
  16. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 287
  17. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
  18. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  19. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 239
  20. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  21. 2023OccWorld: Learning a 3D Occupancy World Model for Autonomous Drivingcited by 230
  22. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  23. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  24. 2024GenAD: Generative End-to-End Autonomous Drivingcited by 225
  25. 2023Scene as Occupancycited by 208
  26. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
  27. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  28. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  29. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  30. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  31. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
  32. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  33. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
  34. 2023FB-OCC: 3D Occupancy Prediction based on Forward-Backward View Transformationcited by 162
  35. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  36. 2024Enhancing End-to-End Autonomous Driving with Latent World Modelcited by 161
  37. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  38. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Modelscited by 158

50 most cited of 1,213 All 1,213 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021BEV-MODNet: Monocular Camera based Bird's Eye View Moving Object Detection for Autonomous Driving
  2. 2025BEVTraj: Map-Free End-to-End Trajectory Prediction in Bird's-Eye View with Deformable Attention and Sparse Goal Proposals
  3. 2025Progressive Bird's Eye View Perception for Safety-Critical Autonomous Driving: A Comprehensive Survey
  4. 2026TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Driving
  5. 2026Planning-Aligned Pretraining of BEV Representations with Sparse Action-Conditioned Targets for End-to-End Autonomous Driving
  6. 2022NMR: Neural Manifold Representation for Autonomous Driving
  7. 2026Closed-Loop Evaluation of Bird's-Eye-View Maps from Cross-View Transformers as Inputs to Behavior-Cloning Policies
  8. 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scale