mlsanityData to 14 Oct 2026

3D-MOOD: Lifting 2D to 3D for Monocular Open-Set Object Detection

arXiv 2507.23567 ↗First posted 31 Jul 2025cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Monocular 3D object detection is valuable for various applications such as robotics and AR/VR. Existing methods are confined to closed-set settings, where the training and testing sets consist of the same scenes and/or object categories. However, real-world applications often introduce new environments and novel object categories, posing a challenge to these methods. In this paper, we address monocular 3D object detection in an open-set setting and introduce the first end-to-end 3D Monocular Open-set Object Detector (3D-MOOD). We propose to lift the open-set 2D detection into 3D space through our designed 3D bounding box head, enabling end-to-end joint training for both 2D and 3D tasks to yield better overall performance. We condition the object queries with geometry prior and overcome the generalization for 3D estimation across diverse scenes. To further improve performance, we design the canonical image space for more efficient cross-dataset training. We evaluate 3D-MOOD on both closed-set settings (Omni3D) and open-set settings (Omni3D to Argoverse 2, ScanNet), and achieve new state-of-the-art results. Code and models are available at royyang0714.github.io/3D-MOOD.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 54 of its 68 references that are in the map. First posted on arXiv 31 Jul 2025.

Builds on

54 of 68 references.

References · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2023Segment Anythingcited by 8,234
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  10. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  11. 2022A ConvNet for the 2020scited by 3,579
  12. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
38 more
  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
  4. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  5. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  7. 2021Grounded Language-Image Pre-trainingcited by 1,156
  8. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  9. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  10. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  11. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  12. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  13. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  14. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  15. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  16. 2017Deep Layer Aggregationcited by 436
  17. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  18. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  19. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
  20. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
  21. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  22. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 249
  23. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  24. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 204
  25. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 191
  26. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 173
  27. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  28. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 160
  29. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  30. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  31. 2022MonoDETR: Depth-guided Transformer for Monocular 3D Object Detectioncited by 86
  32. 2021Monocular Quasi-Dense 3D Object Trackingcited by 81
  33. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 70
  34. 2024An Open and Comprehensive Pipeline for Unified Object Grounding and Detectioncited by 65
  35. 2019Does computer vision matter for action?cited by 64
  36. 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 62
  37. 2021Towards Real-Time Monocular Depth Estimation for Robotics: A Surveycited by 57
  38. 2024Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Communitycited by 50

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.

  1. 2026WildDet3D: Scaling Promptable 3D Detection in the Wildcited by 14
  2. 2026Thinking with Spatial Code for Physical-World Video Reasoningcited by 7
  3. 2026Boxer: Robust Lifting of Open-World 2D Bounding Boxes to 3Dcited by 7
  4. 2025LeAD-M3D: Leveraging Asymmetric Distillation for Real-Time Monocular 3D Detectioncited by 1
  5. 2026Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputscited by 1
  6. 2025LAA3D: A Benchmark of Detecting and Tracking Low-Altitude Aircraft in 3D Spacecited by 0
  7. 2026RAD: Retrieval-Augmented Monocular Metric Depth Estimation for Underrepresented Classescited by 0
  8. 2026MoCA3D: Monocular 3D Bounding Box Prediction in the Image Planecited by 0
  9. 2026SpatialPoint: Spatial-aware Point Prediction for Embodied Localizationcited by 0
  10. 2026RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detectioncited by 0
  11. 2026OmniAct3D: Leveraging Foundation Geometry and Evidence-Grounded Reasoning for Panoramic 3D Detectioncited by 0