mlsanityData to 7 Oct 2026

Orthographic Feature Transform for Monocular 3D Object Detection

arXiv 1811.08188 ↗First posted 20 Nov 2018cs.CVcited by 201

Not on the map. Why?

Abstract (arXiv)

3D object detection from monocular images has proven to be an enormously challenging task, with the performance of leading systems not yet achieving even 10% of that of LiDAR-based counterparts. One explanation for this performance gap is that existing systems are entirely at the mercy of the perspective image-based representation, in which the appearance and scale of objects varies drastically with depth and meaningful distances are difficult to infer. In this work we argue that the ability to reason about the world in 3D is an essential element of the 3D object detection task. To this end, we introduce the orthographic feature transform, which enables us to escape the image domain by mapping image-based features into an orthographic 3D space. This allows us to reason holistically about the spatial configuration of the scene in a domain where scale is consistent and distances between objects are meaningful. We apply this transformation as part of an end-to-end deep learning architecture and achieve state-of-the-art performance on the KITTI 3D object benchmark.footnoteWe will release full source code and pretrained models upon acceptance of this manuscript for publication.

Key facts. Cited by 201 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 38; 2022: 60; 2023: 62; 2024: 23; 2025: 14; 2026: 4.

201 papers.

Sort:

By area:

  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  2. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  3. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  4. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  5. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  6. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 464
  7. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  8. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  9. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  10. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  11. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  12. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
38 more
  1. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  2. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  3. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  4. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  5. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  6. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  7. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  8. 2021Geometry Uncertainty Projection Network for Monocular 3D Object Detectioncited by 155
  9. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 155
  10. 2023V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perceptioncited by 155
  11. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 149
  12. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  13. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  14. 2021Delving into Localization Errors for Monocular 3D Object Detectioncited by 138
  15. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  16. 2022Vision-Centric BEV Perception: A Surveycited by 126
  17. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  18. 2021Translating Images into Mapscited by 115
  19. 2022MonoDTR: Monocular 3D Object Detection with Depth-Aware Transformercited by 108
  20. 20213D Object Detection for Autonomous Driving: A Surveycited by 106
  21. 2023FB-BEV: BEV Representation from Forward-Backward View Transformationscited by 98
  22. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  23. 2021Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detectioncited by 84
  24. 2021Geometry-based Distance Decomposition for Monocular 3D Object Detectioncited by 83
  25. 2023Collaboration Helps Camera Overtake LiDAR in 3D Detectioncited by 81
  26. 2022Efficient and Robust 2D-to-BEV Representation Learning via Geometry-guided Kernel Transformercited by 80
  27. 2023UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representationcited by 74
  28. 2022MonoDistill: Learning Spatial Features for Monocular 3D Object Detectioncited by 68
  29. 20223D Object Detection from Images for Autonomous Driving: A Surveycited by 66
  30. 2023FocalFormer3D : Focusing on Hard Instance for 3D Object Detectioncited by 58
  31. 2023Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Reviewcited by 57
  32. 2023LXL: LiDAR Excluded Lean 3D Object Detection with 4D Imaging Radar and Camera Fusioncited by 57
  33. 2024Context and Geometry Aware Voxel Transformer for Semantic Scene Completioncited by 50
  34. 2022Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detectioncited by 49
  35. 2021Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Imagescited by 48
  36. 2023SMURF: Spatial Multi-Representation Fusion for 3D Object Detection with 4D Imaging Radarcited by 48
  37. 2022NeRF-RPN: A general framework for object detection in NeRFscited by 46
  38. 2022Learning Ego 3D Representation as Ray Tracingcited by 43

50 most cited of 201 All 201 (JSON)