mlsanityData to 7 Oct 2026

DETR4D: Direct Multi-View 3D Object Detection with Sparse Attention

arXiv 2212.07849 ↗First posted 15 Dec 2022cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

3D object detection with surround-view images is an essential task for autonomous driving. In this work, we propose DETR4D, a Transformer-based framework that explores sparse attention and direct feature query for 3D object detection in multi-view images. We design a novel projective cross-attention mechanism for query-image interaction to address the limitations of existing methods in terms of geometric cue exploitation and information loss for cross-view objects. In addition, we introduce a heatmap generation technique that bridges 3D and 2D spaces efficiently via query initialization. Furthermore, unlike the common practice of fusing intermediate spatial features for temporal aggregation, we provide a new perspective by introducing a novel hybrid approach that performs cross-frame fusion over past object queries and image features, enabling efficient and robust modeling of temporal information. Extensive experiments on the nuScenes dataset demonstrate the effectiveness and efficiency of the proposed DETR4D.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 45 of its 50 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

45 of 50 references.

References · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  9. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  10. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  11. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  12. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
33 more
  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  2. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  3. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  4. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  5. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  6. 2017Focal Loss for Dense Object Detectioncited by 749
  7. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  8. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  9. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  10. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  11. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  12. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 486
  13. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  14. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  15. 2021Conditional DETR for Fast Training Convergencecited by 422
  16. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  17. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  18. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  19. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  20. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  21. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  22. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  23. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  24. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  25. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  26. 2021Efficient DETR: Improving End-to-End Object Detector with Dense Priorcited by 128
  27. 2019Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstructioncited by 106
  28. 2021Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsitycited by 98
  29. 2020A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird's Eye Viewcited by 71
  30. 2021Direct Multi-view Multi-person 3D Pose Estimationcited by 64
  31. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  32. 2022Meta-DETR: Image-Level Few-Shot Detection with Inter-Class Correlation Exploitationcited by 48
  33. 2018The Right (Angled) Perspective: Improving the Understanding of Road Scenes Using Boosted Inverse Perspective Mappingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 6; 2026: 2.

21 papers.

Sort:

By area:

  1. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  2. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  3. 2023Online Map Vectorization for Autonomous Driving: A Rasterization Perspectivecited by 54
  4. 2024FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective Viewcited by 51
  5. 2023BEVHeight++: Toward Robust Visual Centric 3D Object Detectioncited by 29
  6. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  7. 2024SeaBird: Segmentation in Bird's View with Dice Loss Improves Monocular 3D Detection of Large Objectscited by 12
  8. 2023OCBEV: Object-Centric BEV Transformer for Multi-View 3D Object Detectioncited by 10
  9. 2024SimPB: A Single Model for 2D and 3D Object Detection from Multiple Camerascited by 9
  10. 2024DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representationcited by 9
  11. 2024Learning High-resolution Vector Representation from Multi-Camera Images for 3D Object Detectioncited by 4
  12. 2026SparseLaneSTP: Leveraging Spatio-Temporal Priors with Sparse Transformers for 3D Lane Detectioncited by 4
9 more
  1. 2024EVT: Efficient View Transformation for Multi-Modal 3D Object Detectioncited by 2
  2. 2025Bridging Text and Video Generation: A Surveycited by 2
  3. 2024AdaOcc: Adaptive-Resolution Occupancy Predictioncited by 1
  4. 2025SimBEV: A Synthetic Multi-Task Multi-Sensor Driving Data Generation Tool and Datasetcited by 1
  5. 2025S2GO: Streaming Sparse Gaussian Occupancy Predictioncited by 1
  6. 2025MambaMap: Online Vectorized HD Map Construction using State Space Modelcited by 1
  7. 2025LiteFusion: Taming 3D Object Detectors from Vision-Based to Multi-Modal with Minimal Adaptationcited by 1
  8. 2025OnlineBEV: Recurrent Temporal Fusion in Bird's Eye View Representations for Multi-Camera 3D Perceptioncited by 0
  9. 2026SimPB++: Simultaneously Detecting 2D and 3D Objects from Multiple Camerascited by 0