mlsanityData to 14 Oct 2026

DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries

arXiv 2110.06922 ↗First posted 13 Oct 2021cs.CV cs.AI cs.LGcited by 658

Not on the map. Why?

Cited by 55 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce a framework for multi-camera 3D object detection. In contrast to existing works, which estimate 3D bounding boxes directly from monocular images or use depth prediction networks to generate input for 3D object detection from 2D information, our method manipulates predictions directly in 3D space. Our architecture extracts 2D features from multiple camera images and then uses a sparse set of 3D object queries to index into these 2D features, linking 3D positions to multi-view images using camera transformation matrices. Finally, our model makes a bounding box prediction per object query, using a set-to-set loss to measure the discrepancy between the ground-truth and the prediction. This top-down approach outperforms its bottom-up counterpart in which object bounding box prediction follows per-pixel depth estimation, since it does not suffer from the compounding error introduced by a depth prediction model. Moreover, our method does not require post-processing such as non-maximum suppression, dramatically improving inference speed. We achieve state-of-the-art performance on the nuScenes autonomous driving benchmark.

Key facts. Cited by 658 papers in the map (data through 14 Oct 2026). Builds on 34 of its 42 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

34 of 42 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2017Mask R-CNNcited by 7,277
  6. 2016Layer Normalizationcited by 5,669
  7. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  8. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  9. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  10. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  11. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  12. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
22 more
  1. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  2. 2017Deformable Convolutional Networkscited by 1,488
  3. 2019Objects as Pointscited by 1,175
  4. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  5. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 495
  6. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  7. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  8. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 439
  9. 2017Deep Layer Aggregationcited by 436
  10. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 382
  11. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  12. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 278
  13. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  14. 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
  15. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 204
  16. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 202
  17. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 191
  18. 2020Rethinking Transformer-based Set Prediction for Object Detectioncited by 156
  19. 2015End-to-end people detection in crowded scenescited by 140
  20. 2019Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstructioncited by 106
  21. 2019Monocular 3D Object Detection via Geometric Reasoning on Keypointscited by 31
  22. 2020Monocular Differentiable Rendering for Self-Supervised 3D Object Detectioncited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 83; 2023: 168; 2024: 175; 2025: 131; 2026: 98.

658 papers.

Sort:

By area:

  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  2. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
  3. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  4. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  5. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  6. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  7. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  8. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  9. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
  10. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  11. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
  12. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
38 more
  1. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  2. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
  3. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 315
  4. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  5. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  6. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  7. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 245
  8. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  9. 2022FUTR3D: A Unified Sensor Fusion Framework for 3D Detectioncited by 194
  10. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 193
  11. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  12. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 178
  13. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
  14. 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 168
  15. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
  16. 2023A Language Agent for Autonomous Drivingcited by 164
  17. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  18. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 156
  19. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  20. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  21. 2022DETRs with Hybrid Matchingcited by 150
  22. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 150
  23. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  24. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoningcited by 143
  25. 2022PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmarkcited by 140
  26. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  27. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 128
  28. 2022Vision-Centric BEV Perception: A Surveycited by 126
  29. 2022DeepInteraction: 3D Object Detection via Modality Interactioncited by 125
  30. 2022BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigscited by 123
  31. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  32. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 122
  33. 2021A Survey of Visual Transformerscited by 117
  34. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 117
  35. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 115
  36. 2023Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Drivingcited by 114
  37. 2023Language Prompt for Autonomous Drivingcited by 111
  38. 2023ADriver-I: A General World Model for Autonomous Drivingcited by 104

50 most cited of 658 All 658 (JSON)