mlsanityData to 7 Oct 2026

BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection

arXiv 2203.17054 ↗First posted 31 Mar 2022cs.CVcited by 340

Not on the map. Why?

Cited by 23 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Single frame data contains finite information which limits the performance of the existing vision-based multi-camera 3D object detection paradigms. For fundamentally pushing the performance boundary in this area, a novel paradigm dubbed BEVDet4D is proposed to lift the scalable BEVDet paradigm from the spatial-only 3D space to the spatial-temporal 4D space. We upgrade the naive BEVDet framework with a few modifications just for fusing the feature from the previous frame with the corresponding one in the current frame. In this way, with negligible additional computing budget, we enable BEVDet4D to access the temporal cues by querying and comparing the two candidate features. Beyond this, we simplify the task of velocity prediction by removing the factors of ego-motion and time in the learning target. As a result, BEVDet4D with robust generalization performance reduces the velocity error by up to -62.9%. This makes the vision-based methods, for the first time, become comparable with those relied on LiDAR or radar in this aspect. On challenge benchmark nuScenes, we report a new record of 54.5% NDS with the high-performance configuration dubbed BEVDet4D-Base, which surpasses the previous leading method BEVDet-Base by +7.3% NDS. The source code is publicly available for further research at https://github.com/HuangJunJie2017/BEVDet .

Key facts. Cited by 340 papers in the map (data through 7 Oct 2026). Builds on 42 of its 55 references that are in the map. First posted on arXiv 31 Mar 2022.

Builds on

42 of 55 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  7. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  8. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  9. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  10. 2019Objects as Pointscited by 1,172
  11. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  12. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
30 more
  1. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  2. 2016Training Deep Nets with Sublinear Memory Costcited by 825
  3. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  4. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  5. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  6. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  7. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  8. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  9. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  10. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  11. 2019Disentangling Monocular 3D Object Detectioncited by 215
  12. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  13. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 184
  14. 2017Flow-Guided Feature Aggregation for Video Object Detectioncited by 177
  15. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  16. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  17. 2020CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detectioncited by 159
  18. 2021Geometry Uncertainty Projection Network for Monocular 3D Object Detectioncited by 155
  19. 2020Memory Enhanced Global-Local Aggregation for Video Object Detectioncited by 128
  20. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  21. 2021AutoShape: Real-Time Shape-Aware Monocular 3D Object Detectioncited by 87
  22. 2021Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detectioncited by 84
  23. 2019Relation Distillation Networks for Video Object Detectioncited by 77
  24. 2021Monocular 3D Object Detection: An Extrinsic Parameter Free Approachcited by 69
  25. 2021GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detectioncited by 66
  26. 2017Towards High Performance Video Object Detectioncited by 60
  27. 2021Progressive Coordinate Transforms for Monocular 3D Object Detectioncited by 52
  28. 2017Mobile Video Object Detection with Temporally-Aware Feature Mapscited by 39
  29. 2021The Devil is in the Task: Exploiting Reciprocal Appearance-Localization Features for Monocular 3D Object Detectioncited by 34
  30. 2019Looking Fast and Slow: Memory-Guided Mobile Video Object Detectioncited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 35; 2023: 79; 2024: 116; 2025: 71; 2026: 39.

340 papers.

Sort:

By area:

  1. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  2. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  3. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  4. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  5. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  6. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  7. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 239
  8. 2023Scene as Occupancycited by 208
  9. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  10. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  11. 2022DETRs with Hybrid Matchingcited by 149
  12. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 149
38 more
  1. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  2. 2023StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Constructioncited by 143
  3. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  4. 2022Vision-Centric BEV Perception: A Surveycited by 126
  5. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  6. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  7. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
  8. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 92
  9. 2022BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereocited by 89
  10. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Drivingcited by 89
  11. 2023SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detectioncited by 83
  12. 2024RCBEVDet: Radar-camera Fusion in Bird's Eye View for 3D Object Detectioncited by 82
  13. 2023Collaboration Helps Camera Overtake LiDAR in 3D Detectioncited by 81
  14. 2022BEVPoolv2: A Cutting-edge Implementation of BEVDet Toward Deploymentcited by 79
  15. 2023BEVHeight: A Robust Framework for Vision-based Roadside 3D Object Detectioncited by 74
  16. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  17. 2023Far3D: Expanding the Horizon for Surround-view 3D Object Detectioncited by 70
  18. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69
  19. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Drivingcited by 67
  20. 2024MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mappingcited by 67
  21. 2022BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detectioncited by 66
  22. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Predictioncited by 66
  23. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  24. 2023Fully Sparse 3D Occupancy Predictioncited by 61
  25. 2022STS: Surround-view Temporal Stereo for Multi-view 3D Detectioncited by 53
  26. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  27. 2023Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Predictioncited by 49
  28. 2023OctreeOcc: Efficient and Multi-Granularity Occupancy Prediction Using Octree Queriescited by 48
  29. 2022Bridging the View Disparity Between Radar and Camera Features for Multi-modal Fusion 3D Object Detectioncited by 46
  30. 2024RenderWorld: World Model with Self-Supervised 3D Labelcited by 44
  31. 2024Stream Query Denoising for Vectorized HD Map Constructioncited by 43
  32. 2023DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillationcited by 41
  33. 2023Fast-BEV: A Fast and Strong Bird's-Eye View Perception Baselinecited by 40
  34. 2023CAPE: Camera View Position Embedding for Multi-View 3D Object Detectioncited by 39
  35. 2024Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Drivingcited by 39
  36. 2023Object as Query: Lifting any 2D Object Detector to 3D Detectioncited by 38
  37. 2023EA-LSS: Edge-aware Lift-splat-shot Framework for 3D BEV Object Detectioncited by 38
  38. 2022TiG-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learningcited by 37

50 most cited of 340 All 340 (JSON)