mlsanityData to 14 Oct 2026

Auto-Vocabulary 3D Object Detection

arXiv 2512.16077 ↗First posted 18 Dec 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Open-vocabulary 3D object detection methods are able to localize 3D boxes of classes unseen during training. Despite the name, existing methods rely on user-specified classes both at training and inference. We propose to study Auto-Vocabulary 3D Object Detection (AV3DOD), where the classes are automatically generated for the detected objects without any user input. To this end, we introduce Semantic Score (SS) to evaluate the quality of the generated class names. We then develop a novel framework, AV3DOD, which leverages 2D vision-language models (VLMs) to generate rich semantic candidates through image captioning, pseudo 3D box generation, and feature-space semantics expansion. AV3DOD achieves the state-of-the-art (SOTA) performance on both localization (mAP) and semantic quality (SS) on the ScanNetV2 and SUNRGB-D datasets. Notably, it surpasses the SOTA, CoDA, by 3.48 overall mAP and attains a 24.5% relative improvement in SS on ScanNetV2.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 60 of its 72 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

60 of 72 references.

References · 60

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  6. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  7. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  8. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  9. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  10. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  11. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  12. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 246
38 more
  1. 2021Group-Free 3D Object Detection via Transformerscited by 215
  2. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  3. 2021Delving into Localization Errors for Monocular 3D Object Detectioncited by 140
  4. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  5. 2020H3DNet: 3D Object Detection Using Hybrid Geometric Primitivescited by 123
  6. 2023Open-Vocabulary Point-Cloud Object Detection without 3D Annotationcited by 81
  7. 2020MLCVNet: Multi-Level Context VoteNet for 3D Object Detectioncited by 80
  8. 2023CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Datacited by 69
  9. 2021Back-tracing Representative Points for Voting-based 3D Object Detection in Point Cloudscited by 67
  10. 2023CoDA: Collaborative Novel Box Discovery and Cross-modal Alignment for Open-vocabulary 3D Object Detectioncited by 52
  11. 2024MonoCD: Monocular 3D Object Detection with Complementary Depthscited by 39
  12. 2020InfoFocus: 3D Object Detection for Autonomous Driving with Dynamic Information Modelingcited by 35
  13. 2023Vocabulary-free Image Classificationcited by 35
  14. 2025DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perceptioncited by 33
  15. 2024InstaGen: Enhancing Object Detection by Training on Synthetic Datasetcited by 26
  16. 2023Taming Self-Training for Open-Vocabulary Object Detectioncited by 25
  17. 2024Learning Background Prompts to Discover Implicit Knowledge for Open Vocabulary Object Detectioncited by 24
  18. 2023Toward Open Vocabulary Aerial Object Detection with CLIP-Activated Student-Teacher Learningcited by 22
  19. 2023OpenSight: A Simple Open-Vocabulary Framework for LiDAR-Based Object Detectioncited by 22
  20. 2024Find n' Propagate: Open-Vocabulary 3D Object Detection in Urban Environmentscited by 21
  21. 2019Embodied Language Grounding with 3D Visual Feature Representationscited by 20
  22. 2024Retrieval-Augmented Open-Vocabulary Object Detectioncited by 20
  23. 2024OV-Uni3DETR: Towards Unified Open-Vocabulary 3D Object Detection via Cycle-Modality Propagationcited by 19
  24. 2024LaMI-DETR: Open-Vocabulary Detection with Language Model Instructioncited by 19
  25. 2024Collaborative Novel Object Discovery and Box-Guided Cross-Modal Alignment for Open-Vocabulary 3D Object Detectioncited by 17
  26. 2024ImOV3D: Learning Open-Vocabulary Point Clouds 3D Object Detection from Only 2D Imagescited by 17
  27. 2024MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoderscited by 16
  28. 2024Unlocking Textual and Visual Wisdom: Open-Vocabulary 3D Object Detection Enhanced by Comprehensive Guidance from Text and Imagecited by 14
  29. 2024Hyperbolic Learning with Synthetic Captions for Open-World Detectioncited by 12
  30. 2024Decoupled Pseudo-labeling for Semi-Supervised Monocular 3D Object Detectioncited by 11
  31. 2024SHiNe: Semantic Hierarchy Nexus for Open-vocabulary Object Detectioncited by 11
  32. 2024MarvelOVD: Marrying Object Recognition and Vision-Language Models for Robust Open-Vocabulary Object Detectioncited by 11
  33. 2019A 3D-Deep-Learning-based Augmented Reality Calibration Method for Robotic Environments using Depth Sensor Datacited by 10
  34. 2024Global-Local Collaborative Inference with LLM for Lidar-Based Open-Vocabulary Detectioncited by 10
  35. 2024MonoWAD: Weather-Adaptive Diffusion Model for Robust Monocular 3D Object Detectioncited by 10
  36. 2025Learning Class Prototypes for Unified Sparse Supervised 3D Object Detectioncited by 10
  37. 2023Weakly Supervised 3D Object Detection via Multi-Level Visual Guidancecited by 9
  38. 2025Cyclic Contrastive Knowledge Transfer for Open-Vocabulary Object Detectioncited by 9

50 most cited of 60 All 60 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Just Add Geometry: Gradient-Free Open-Vocabulary 3D Detection Without Human-in-the-Loop
  2. 2025HQ-OV3D: A High Box Quality Open-World 3D Detection Framework based on Diffision Model
  3. 2025LocateAnything3D: Vision-Language 3D Detection with Chain-of-Sight
  4. 2025Hierarchical Cross-Modal Alignment for Open-Vocabulary 3D Object Detection
  5. 2025A Review of 3D Object Detection with Vision-Language Models
  6. 2026Group3D: MLLM-Driven Semantic Grouping for Open-Vocabulary 3D Object Detection
  7. 2024Open-Vocabulary SAM3D: Towards Training-free Open-Vocabulary 3D Scene Understanding
  8. 2023OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation