mlsanityData to 14 Oct 2026

SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Grounding

arXiv 2412.04383 ↗First posted 5 Dec 2024cs.CV cs.ROcited by 67

Not on the map. Why?

Cited by 20 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

3D Visual Grounding (3DVG) aims to locate objects in 3D scenes based on textual descriptions, essential for applications like augmented reality and robotics. Traditional 3DVG approaches rely on annotated 3D datasets and predefined object categories, limiting scalability and adaptability. To overcome these limitations, we introduce SeeGround, a zero-shot 3DVG framework leveraging 2D Vision-Language Models (VLMs) trained on large-scale 2D data. SeeGround represents 3D scenes as a hybrid of query-aligned rendered images and spatially enriched text descriptions, bridging the gap between 3D data and 2D-VLMs input formats. We propose two modules: the Perspective Adaptation Module, which dynamically selects viewpoints for query-relevant image rendering, and the Fusion Alignment Module, which integrates 2D images with 3D spatial descriptions to enhance object localization. Extensive experiments on ScanRefer and Nr3D demonstrate that our approach outperforms existing zero-shot methods by large margins. Notably, we exceed weakly supervised methods and rival some fully supervised ones, outperforming previous SOTA by 7.7% on ScanRefer and 7.1% on Nr3D, showcasing its effectiveness in complex 3DVG tasks.

Key facts. Cited by 67 papers in the map (data through 14 Oct 2026). Builds on 62 of its 72 references that are in the map. First posted on arXiv 5 Dec 2024.

Builds on

62 of 72 references.

References · 62

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  7. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  8. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  9. 2023LERF: Language Embedded Radiance Fieldscited by 543
  10. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  11. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  12. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
38 more
  1. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 303
  2. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 289
  3. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 239
  4. 2024CogVLM2: Visual Language Models for Image and Video Understandingcited by 217
  5. 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 204
  6. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 181
  7. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  8. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 166
  9. 2023EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AIcited by 147
  10. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 141
  11. 2022Multi-View Transformer for 3D Visual Groundingcited by 140
  12. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agentcited by 135
  13. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 133
  14. 2023Robo3D: Towards Robust and Reliable 3D Perception against Corruptionscited by 129
  15. 2021Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Cloudscited by 128
  16. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 128
  17. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 125
  18. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  19. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 122
  20. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 99
  21. 2023RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understandingcited by 94
  22. 2023SAI3D: Segment Any Instance in 3D Scenescited by 90
  23. 2023OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentationcited by 82
  24. 2023OVIR-3D: Open-Vocabulary 3D Instance Retrieval Without Training on 3D Datacited by 81
  25. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70
  26. 2024Unifying 3D Vision-Language Understanding via Promptable Queriescited by 70
  27. 2023Visual Programming for Zero-shot Open-Vocabulary 3D Visual Groundingcited by 64
  28. 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
  29. 2024VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Groundingcited by 56
  30. 2022Look Around and Refer: 2D Synthetic Semantics Knowledge Distillation for 3D Visual Groundingcited by 41
  31. 2024PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulationcited by 36
  32. 20244D Contrastive Superflows are Dense 3D Representation Learnerscited by 33
  33. 2023Four Ways to Improve Verbo-visual Fusion for Dense 3D Visual Groundingcited by 32
  34. 2024MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Groundingcited by 29
  35. 2024Agent3D-Zero: An Agent for Zero-shot 3D Understandingcited by 29
  36. 2023Interactive Planning Using Large Language Models for Partially Observable Robotics Taskscited by 27
  37. 2024Is Your LiDAR Placement Optimized for 3D Scene Understanding?cited by 26
  38. 2024Multi-branch Collaborative Learning Network for 3D Visual Groundingcited by 20

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 33; 2026: 33.

67 papers.

Sort:

By area:

  1. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 118
  2. 2024Toward Visual Grounding: A Surveycited by 52
  3. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  4. 2025N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Modelscited by 29
  5. 2025Reasoning in Space via Grounding in the Worldcited by 19
  6. 2025AffordBot: 3D Fine-grained Embodied Reasoning via Multimodal Large Language Modelscited by 15
  7. 2025Scene-R1: Video-Grounded Large Language Models for 3D Scene Reasoning without 3D Annotationscited by 14
  8. 2025SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Groundingcited by 12
  9. 20253EED: Ground Everything Everywhere in 3Dcited by 12
  10. 2025SORT3D: Spatial Object-centric Reasoning Toolbox for Zero-Shot 3D Grounding Using Large Language Modelscited by 11
  11. 2025SPIRAL: Semantic-Aware Progressive LiDAR Scene Generation and Understandingcited by 10
  12. 2025Language-to-Space Programming for Training-Free 3D Visual Groundingcited by 8
38 more
  1. 2025SIRI-Bench: Challenging VLMs' Spatial Intelligence through Complex Reasoning Taskscited by 7
  2. 2025Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representationscited by 7
  3. 2025SeqVLM: Proposal-Guided Multi-View Sequences Reasoning via VLM for Zero-Shot 3D Visual Groundingcited by 7
  4. 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 7
  5. 2025Talk2Event: Grounded Understanding of Dynamic Scenes from Event Camerascited by 6
  6. 20254DLangVGGT: 4D Language-Visual Geometry Grounded Transformercited by 5
  7. 2025Global-Aware Monocular Semantic Scene Completion with State Space Modelscited by 3
  8. 2025MineAnyBuild: Benchmarking Spatial Planning for Open-world AI Agentscited by 3
  9. 2025Warehouse Spatial Question Answering with LLM Agentcited by 3
  10. 2025MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Imagescited by 3
  11. 2025PanoGrounder: Bridging 2D and 3D with Panoramic Scene Representations for VLM-based 3D Visual Groundingcited by 3
  12. 2026AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environmentscited by 3
  13. 20263D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decodingcited by 3
  14. 2025Zero-Shot 3D Visual Grounding from Vision-Language Modelscited by 2
  15. 2025FreeQ-Graph: Free-form Querying with Semantic Consistent Scene Graph for 3D Scene Understandingcited by 2
  16. 2026Z3D: Zero-Shot 3D Visual Grounding from Imagescited by 2
  17. 2026Point Cloud as a Foreign Language for Multi-modal Large Language Modelcited by 2
  18. 2026OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generationcited by 2
  19. 2026EventDrive: Event Cameras for Vision-Language Driving Intelligencecited by 2
  20. 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
  21. 2025T-Rex: Task-Adaptive Spatial Representation Extraction for Robotic Manipulation with Vision-Language Modelscited by 1
  22. 2025Visual Grounding from Event Camerascited by 1
  23. 2025ChangingGrounding: 3D Visual Grounding in Changing Scenescited by 1
  24. 2025Error-Driven Scene Editing for 3D Grounding in Large Language Modelscited by 1
  25. 2026VIZOR: Viewpoint-Invariant Zero-Shot Scene Graph Generation for 3D Scene Reasoningcited by 1
  26. 20263D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understandingcited by 1
  27. 2026UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsingcited by 1
  28. 2026Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Groundingcited by 1
  29. 2026THOM: Generating Physically Plausible Hand-Object Meshes From Textcited by 1
  30. 2026PanopticQuery: Unified Query-Time Reasoning for 4D Scenescited by 1
  31. 2026TAIHRI: Task-Aware 3D Human Keypoints Localization for Close-Range Human-Robot Interactioncited by 1
  32. 2026Perceive-then-Plan: Layout-as-Policy for Monocular 3D Scene Layout Estimationcited by 1
  33. 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?cited by 1
  34. 2025DSM: Constructing a Diverse Semantic Map for 3D Visual Groundingcited by 0
  35. 2025Learning to Navigate Socially Through Proactive Risk Perceptioncited by 0
  36. 2025Thinking Ahead: Foresight Intelligence in MLLMs and World Modelcited by 0
  37. 2025TrackTeller: Temporal Multimodal 3D Grounding for Behavior-Dependent Object Referencescited by 0
  38. 2025OpenGround: Planning-based Online Perception for Open-World 3D Visual Groundingcited by 0

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025View-on-Graph: Zero-shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs
  2. 2026TDVR: Joint Text Disambiguation and Viewpoint Reasoning for Zero-Shot 3D Visual Grounding
  3. 2026GuideGround: VLM-guided Semantic Understanding and Viewpoint-aware Reasoning for 3D Visual Grounding
  4. 2024Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction Problems
  5. 2025From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes
  6. 2023Weakly-Supervised 3D Visual Grounding based on Visual Language Alignment
  7. 2026Multiple Consistent 2D-3D Mappings for Robust Zero-Shot 3D Visual Grounding
  8. 2026Grounded 3D-Aware Spatial Vision-Language Modeling