# MonoDETR: Depth-guided Transformer for Monocular 3D Object Detection

arXiv: https://arxiv.org/abs/2203.13310 · HTML: https://mlsanity.com/paper/2203.13310/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 72 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 30 of 58)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection](https://mlsanity.com/paper/2104.10956/) (2021)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [Soft-NMS -- Improving Object Detection With One Line of Code](https://mlsanity.com/paper/1704.04503/) (2017)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)

## Built on by (most cited first, 30 of 86)

- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [Query-Dependent Video Representation for Moment Retrieval and Highlight Detection](https://mlsanity.com/paper/2303.13874/) (2023)
- [OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2412.15208/) (2024)
- [Far3D: Expanding the Horizon for Surround-view 3D Object Detection](https://mlsanity.com/paper/2308.09616/) (2023)
- [Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation](https://mlsanity.com/paper/2305.16318/) (2023)
- [LATR: 3D Lane Detection from Monocular Images with Transformer](https://mlsanity.com/paper/2308.04583/) (2023)
- [MonoCD: Monocular 3D Object Detection with Complementary Depths](https://mlsanity.com/paper/2404.03181/) (2024)
- [Detect Anything 3D in the Wild](https://mlsanity.com/paper/2504.07958/) (2025)
- [BEVHeight++: Toward Robust Visual Centric 3D Object Detection](https://mlsanity.com/paper/2309.16179/) (2023)
- [Depth Prompting for Sensor-Agnostic Depth Estimation](https://mlsanity.com/paper/2405.11867/) (2024)
- [Open Vocabulary Monocular 3D Object Detection](https://mlsanity.com/paper/2411.16833/) (2024)
- [MonoGAE: Roadside Monocular 3D Object Detection with Ground-Aware Embeddings](https://mlsanity.com/paper/2310.00400/) (2023)
- [MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors](https://mlsanity.com/paper/2410.19590/) (2024)
- [Learning Cooperative Trajectory Representations for Motion Forecasting](https://mlsanity.com/paper/2311.00371/) (2023)
- [BEV-SAN: Accurate BEV 3D Object Detection via Slice Attention Networks](https://mlsanity.com/paper/2212.01231/) (2022)
- [H3DP: Triply-Hierarchical Diffusion Policy for Visuomotor Learning](https://mlsanity.com/paper/2505.07819/) (2025)
- [WildDet3D: Scaling Promptable 3D Detection in the Wild](https://mlsanity.com/paper/2604.08626/) (2026)
- [DriveGEN: Generalized and Robust 3D Detection in Driving via Controllable Text-to-Image Diffusion Generation](https://mlsanity.com/paper/2503.11122/) (2025)
- [Enhancing 3D Object Detection with 2D Detection-Guided Query Anchors](https://mlsanity.com/paper/2403.06093/) (2024)
- [SeaBird: Segmentation in Bird's View with Dice Loss Improves Monocular 3D Detection of Large Objects](https://mlsanity.com/paper/2403.20318/) (2024)
- [3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection](https://mlsanity.com/paper/2507.23567/) (2025)
- [MonoTAKD: Teaching Assistant Knowledge Distillation for Monocular 3D Object Detection](https://mlsanity.com/paper/2404.04910/) (2024)
- [AerialVG: A Challenging Benchmark for Aerial Visual Grounding by Exploring Positional Relations](https://mlsanity.com/paper/2504.07836/) (2025)
- [Unleashing the Power of Chain-of-Prediction for Monocular 3D Object Detection](https://mlsanity.com/paper/2505.04594/) (2025)
- [RoScenes: A Large-scale Multi-view 3D Dataset for Roadside Perception](https://mlsanity.com/paper/2405.09883/) (2024)
- [WARM-3D: A Weakly-Supervised Sim2Real Domain Adaptation Framework for Roadside Monocular 3D Object Detection](https://mlsanity.com/paper/2407.20818/) (2024)
- [Ev-3DOD: Pushing the Temporal Boundaries of 3D Object Detection with Event Cameras](https://mlsanity.com/paper/2502.19630/) (2025)
- [LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight](https://mlsanity.com/paper/2511.20648/) (2025)
- [CT3D++: Improving 3D Object Detection with Keypoint-induced Channel-wise Transformer](https://mlsanity.com/paper/2406.08152/) (2024)
- [IFTR: An Instance-Level Fusion Transformer for Visual Collaborative Perception](https://mlsanity.com/paper/2407.09857/) (2024)

## Abstract (arXiv)

Monocular 3D object detection has long been a challenging task in autonomous driving. Most existing methods follow conventional 2D detectors to first localize object centers, and then predict 3D attributes by neighboring features. However, only using local visual features is insufficient to understand the scene-level 3D spatial structures and ignores the long-range inter-object depth relations. In this paper, we introduce the first DETR framework for Monocular DEtection with a depth-guided TRansformer, named MonoDETR. We modify the vanilla transformer to be depth-aware and guide the whole detection process by contextual depth cues. Specifically, concurrent to the visual encoder that captures object appearances, we introduce to predict a foreground depth map, and specialize a depth encoder to extract non-local depth embeddings. Then, we formulate 3D object candidates as learnable queries and propose a depth-guided decoder to conduct object-scene depth interactions. In this way, each object query estimates its 3D attributes adaptively from the depth-guided regions on the image and is no longer constrained to local visual features. On KITTI benchmark with monocular images as input, MonoDETR achieves state-of-the-art performance and requires no extra dense depth annotations. Besides, our depth-guided modules can also be plug-and-play to enhance multi-view 3D object detectors on nuScenes dataset, demonstrating our superior generalization capacity. Code is available at https://github.com/ZrrSkywalker/MonoDETR.