mlsanityData to 7 Oct 2026

Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild

arXiv 2207.10660 ↗First posted 21 Jul 2022cs.CVcited by 159

Not on the map. Why?

Cited by 26 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recognizing scenes and objects in 3D from a single image is a longstanding goal of computer vision with applications in robotics and AR/VR. For 2D recognition, large datasets and scalable solutions have led to unprecedented advances. In 3D, existing benchmarks are small in size and approaches specialize in few object categories and specific domains, e.g. urban driving scenes. Motivated by the success of 2D recognition, we revisit the task of 3D object detection by introducing a large benchmark, called Omni3D. Omni3D re-purposes and combines existing datasets resulting in 234k images annotated with more than 3 million instances and 98 categories. 3D detection at such scale is challenging due to variations in camera intrinsics and the rich diversity of scene and object types. We propose a model, called Cube R-CNN, designed to generalize across camera and scene types with a unified approach. We show that Cube R-CNN outperforms prior works on the larger Omni3D and existing benchmarks. Finally, we prove that Omni3D is a powerful dataset for 3D object recognition and show that it improves single-dataset performance and can accelerate learning on new smaller datasets via pre-training.

Key facts. Cited by 159 papers in the map (data through 7 Oct 2026). Builds on 71 of its 96 references that are in the map. First posted on arXiv 21 Jul 2022.

Builds on

71 of 96 references.

References · 71

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2017Mask R-CNNcited by 7,259
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  9. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  11. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  12. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
38 more
  1. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  2. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  3. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  4. 2019Objects as Pointscited by 1,172
  5. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  6. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  7. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  8. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  9. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  10. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  11. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  12. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  13. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  14. 2017Deep Layer Aggregationcited by 436
  15. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  16. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 325
  17. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  18. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  19. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  20. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  21. 2019Disentangling Monocular 3D Object Detectioncited by 215
  22. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 202
  23. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  24. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  25. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  26. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 184
  27. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  28. 2019Learning Depth-Guided Convolutions for Monocular 3D Object Detectioncited by 169
  29. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  30. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  31. 2021Geometry Uncertainty Projection Network for Monocular 3D Object Detectioncited by 155
  32. 2020MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationshipscited by 153
  33. 2020RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Drivingcited by 150
  34. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  35. 2021Delving into Localization Errors for Monocular 3D Object Detectioncited by 138
  36. 2019Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Drivingcited by 137
  37. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  38. 2020Rethinking Pseudo-LiDAR Representationcited by 109

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 26; 2024: 38; 2025: 50; 2026: 45.

159 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  3. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  4. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
  5. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  6. 2023Aria Digital Twin: A New Benchmark Dataset for Egocentric 3D Machine Perceptioncited by 166
  7. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 146
  8. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 133
  9. 2024SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Modelcited by 95
  10. 2025Visual Spatial Tuningcited by 85
  11. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
  12. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
38 more
  1. 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoningcited by 65
  2. 2024Visual Prompting in Multimodal Large Language Models: A Surveycited by 52
  3. 2023NeRF-Det: Learning Geometry-Aware Volumetric Representation for Multi-View 3D Object Detectioncited by 51
  4. 2024Texture-GS: Disentangling the Geometry and Texture for 3D Gaussian Splatting Editingcited by 47
  5. 2023EgoObjects: A Large-Scale Egocentric Dataset for Fine-Grained Object Understandingcited by 39
  6. 2024Cubify Anything: Scaling Indoor 3D Object Detectioncited by 39
  7. 2025Reinforced Visual Perception with Toolscited by 38
  8. 2023Single-view 3D Scene Reconstruction with High-fidelity Shape and Texturecited by 37
  9. 2025SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Modelscited by 37
  10. 2023Learning Occupancy for Monocular 3D Object Detectioncited by 35
  11. 2023Tame a Wild Camera: In-the-Wild Monocular Camera Calibrationcited by 34
  12. 2023Understanding 3D Object Interaction from a Single Imagecited by 33
  13. 20233D Hand Pose Estimation in Everyday Egocentric Imagescited by 33
  14. 2025Detect Anything 3D in the Wildcited by 32
  15. 2024Language-Image Models with 3D Understandingcited by 31
  16. 2024MVInpainter: Learning Multi-View Consistent Inpainting to Bridge 2D and 3D Editingcited by 30
  17. 2023BEVHeight++: Toward Robust Visual Centric 3D Object Detectioncited by 29
  18. 2025N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Modelscited by 28
  19. 2023Uni3DETR: Unified 3D Detection Transformercited by 26
  20. 20233DiffTection: 3D Object Detection with Geometry-Aware Diffusion Featurescited by 26
  21. 2024ImageNet3D: Towards General-Purpose Object-Level 3D Understandingcited by 25
  22. 2024EFM3D: A Benchmark for Measuring Progress Towards 3D Egocentric Foundation Modelscited by 25
  23. 2023OVO: Open-Vocabulary Occupancycited by 24
  24. 2025Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-trainingcited by 23
  25. 2023Mono3DVG: 3D Visual Grounding in Monocular Imagescited by 22
  26. 2024OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objectscited by 22
  27. 2024Open Vocabulary Monocular 3D Object Detectioncited by 22
  28. 2024UniDet3D: Multi-dataset Indoor 3D Object Detectioncited by 20
  29. 2024OV-Uni3DETR: Towards Unified Open-Vocabulary 3D Object Detection via Cycle-Modality Propagationcited by 19
  30. 2025SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligencecited by 19
  31. 2023Generating Images with 3D Annotations Using Diffusion Modelscited by 18
  32. 2024Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generationcited by 18
  33. 2025SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Modelcited by 15
  34. 2025How Well Does GPT-4o Understand Vision? Evaluating Multimodal Foundation Models on Standard Computer Vision Taskscited by 14
  35. 2023Inpaint3D: 3D Scene Content Generation using 2D Inpainting Diffusioncited by 13
  36. 2025Vision-Language Models Can't See the Obviouscited by 13
  37. 2024SeaBird: Segmentation in Bird's View with Dice Loss Improves Monocular 3D Detection of Large Objectscited by 12
  38. 2024CARLA Drone: Monocular 3D Object Detection from a Different Perspectivecited by 12

50 most cited of 159 All 159 (JSON)