mlsanityData to 7 Oct 2026

LabelAny3D: Label Any Object 3D in the Wild

arXiv 2601.01676 ↗First posted 4 Jan 2026cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Detecting objects in 3D space from monocular input is crucial for applications ranging from robotics to scene understanding. Despite advanced performance in the indoor and autonomous driving domains, existing monocular 3D detection models struggle with in-the-wild images due to the lack of 3D in-the-wild datasets and the challenges of 3D annotation. We introduce LabelAny3D, an analysis-by-synthesis framework that reconstructs holistic 3D scenes from 2D images to efficiently produce high-quality 3D bounding box annotations. Built on this pipeline, we present COCO3D, a new benchmark for open-vocabulary monocular 3D detection, derived from the MS-COCO dataset and covering a wide range of object categories absent from existing 3D datasets. Experiments show that annotations generated by LabelAny3D improve monocular 3D detection performance across multiple benchmarks, outperforming prior auto-labeling approaches in quality. These results demonstrate the promise of foundation-model-driven annotation for scaling up 3D recognition in realistic, open-world settings.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 67 of its 78 references that are in the map. First posted on arXiv 4 Jan 2026.

Builds on

67 of 78 references.

References · 67

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2023Segment Anythingcited by 8,158
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  11. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  12. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
38 more
  1. 2024Depth Anything V2cited by 1,385
  2. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  3. 2024Grounding Image Matching in 3D with MASt3Rcited by 960
  4. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  5. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  6. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  7. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  8. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  9. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  10. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  11. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  12. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  13. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  14. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  15. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  16. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 348
  17. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
  18. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  19. 2019Disentangling Monocular 3D Object Detectioncited by 215
  20. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  21. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  22. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  23. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  24. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  25. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  26. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 115
  27. 2022MonoDTR: Monocular 3D Object Detection with Depth-Aware Transformercited by 108
  28. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  29. 2022MonoDETR: Depth-guided Transformer for Monocular 3D Object Detectioncited by 85
  30. 2021MonoRUn: Monocular 3D Object Detection by Reconstruction and Uncertainty Propagationcited by 82
  31. 2023Open-Vocabulary Point-Cloud Object Detection without 3D Annotationcited by 81
  32. 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 59
  33. 2018Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimationcited by 51
  34. 2023CoDA: Collaborative Novel Box Discovery and Cross-modal Alignment for Open-vocabulary 3D Object Detectioncited by 51
  35. 2024Cubify Anything: Scaling Indoor 3D Object Detectioncited by 39
  36. 2021Pseudo-labeling for Scalable 3D Object Detectioncited by 32
  37. 2025Detect Anything 3D in the Wildcited by 32
  38. 2025Dynamic Camera Poses and Where to Find Themcited by 26

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026WildDet3D: Scaling Promptable 3D Detection in the Wildcited by 12
  2. 2026WildLIFT: Lifting monocular drone video to 3D for species-agnostic wildlife monitoringcited by 1
  3. 2026Map-Det3D: Metric Feed-Forward 3D Reconstruction Prior for Multi-view 3D Object Detection from Streaming Inputscited by 1
  4. 2026WildBox: A Dataset and Benchmark for Aerial Monocular 3D Detection of African Savanna Wildlifecited by 0
  5. 2026RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detectioncited by 0
  6. 2026Evaluating 2D and 3D-Aware Vision Foundation Models for Vehicle Attribute Recognitioncited by 0