mlsanityData to 7 Oct 2026

ViGiL3D: A Linguistically Diverse Dataset for 3D Visual Grounding

arXiv 2501.01366 ↗First posted 2 Jan 2025cs.CV cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

3D visual grounding (3DVG) involves localizing entities in a 3D scene referred to by natural language text. Such models are useful for embodied AI and scene retrieval applications, which involve searching for objects or patterns using natural language descriptions. While recent works have focused on LLM-based scaling of 3DVG datasets, these datasets do not capture the full range of potential prompts which could be specified in the English language. To ensure that we are scaling up and testing against a useful and representative set of prompts, we propose a framework for linguistically analyzing 3DVG prompts and introduce Visual Grounding with Diverse Language in 3D (ViGiL3D), a diagnostic dataset for evaluating visual grounding methods against a diverse set of language patterns. We evaluate existing open-vocabulary 3DVG methods to demonstrate that these methods are not yet proficient in understanding and identifying the targets of more challenging, out-of-distribution prompts, toward real-world applications.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 33 of its 71 references that are in the map. First posted on arXiv 2 Jan 2025.

Builds on

33 of 71 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  5. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  6. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  7. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  8. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  9. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
  10. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  11. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 299
  12. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
21 more
  1. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 258
  2. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 142
  3. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 134
  4. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  5. 2021Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Cloudscited by 128
  6. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 121
  7. 2023Encyclopedic VQA: Visual questions about detailed properties of fine-grained categoriescited by 105
  8. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 98
  9. 2024PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Taskscited by 78
  10. 2024Unifying 3D Vision-Language Understanding via Promptable Queriescited by 70
  11. 2022UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Groundingcited by 69
  12. 2023Visual Programming for Zero-shot Open-Vocabulary 3D Visual Groundingcited by 64
  13. 2024ScanReason: Empowering 3D Visual Grounding with Reasoning Capabilitiescited by 47
  14. 2024MMScan: A Multi-Modal 3D Scene Dataset with Hierarchical Grounded Language Annotationscited by 40
  15. 20243D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucinationcited by 36
  16. 2022ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenescited by 32
  17. 2023M3DBench: Let's Instruct Large Models with Multi-modal 3D Promptscited by 32
  18. 2024VLA-3D: A Dataset for 3D Semantic Scene Understanding and Navigationcited by 21
  19. 2024Space3D-Bench: Spatial 3D Question Answering Benchmarkcited by 17
  20. 2023CityRefer: Geography-aware 3D Visual Grounding Dataset on City-scale Point Cloud Datacited by 15
  21. 20243D Question Answering for City Scene Understandingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025OpenLex3D: A Tiered Evaluation Benchmark for Open-Vocabulary 3D Scene Representationscited by 1
  2. 2025MLFM: Multi-Layered Feature Maps for Richer Language Understanding in Zero-Shot Semantic Navigationcited by 1
  3. 2025ChangingGrounding: 3D Visual Grounding in Changing Scenescited by 1
  4. 2025SHREC 2025: Retrieval of Optimal Objects for Multi-modal Enhanced Language and Spatial Assistance (ROOMELSA)cited by 0
  5. 2026Scaling Diverse Language Generation for 3D Visual Groundingcited by 0
  6. 2026SeqAlign3DVG: A Sequence-Aligned Benchmark and Voxel Reasoning Framework for 3D Visual Groundingcited by 0
  7. 2026Task-Adaptive Grounded 3D-Programmers Using 2D VLMscited by 0