mlsanityData to 7 Oct 2026

Where to Look Matters: Learning Influential Views for VLM-based 3D Visual Grounding

arXiv 2609.04741 ↗First posted 4 Sep 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Recent zero-shot 3D visual grounding methods leverage vision-language models (VLMs) to localize objects in 3D scenes from natural language queries. However, these methods typically rely on heuristic rules to select which camera views are provided to the VLM, often prioritizing object visibility rather than grounding relevance. We present IVSGround, a framework that learns Influential View Selection for VLM-based 3D visual grounding. Instead of using fixed heuristics, a lightweight view selector is trained to identify views that provide discriminative evidence for grounding. To obtain supervision signals, we generate training signals using feedback from a reasoning VLM through a two-stage rejection sampling process. During inference, the learned selector predicts query-conditioned influential views for each candidate object, which are then evaluated by a frozen reasoning VLM through comparative grounding. Experiments on ScanRefer and NR3D show that IVSGround consistently improves grounding accuracy over existing zero-shot pipelines, demonstrating that selecting where to look is crucial for effective 3D visual grounding. Project page: https://ivsground.github.io/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 26 of its 29 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

26 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play29 Sep

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  7. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  8. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  9. 2022Multi-View Transformer for 3D Visual Groundingcited by 138
  10. 2021Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Cloudscited by 128
  11. 2025M-LLM Based Video Frame Selection for Efficient Video Understandingcited by 87
  12. 2024Frame-Voyager: Learning to Query Frames for Video Large Language Modelscited by 80
13 more
  1. 2023ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidancecited by 70
  2. 2024SeeGround: See and Ground for Zero-Shot Open-Vocabulary 3D Visual Groundingcited by 66
  3. 2023Visual Programming for Zero-shot Open-Vocabulary 3D Visual Groundingcited by 64
  4. 2024VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Groundingcited by 56
  5. 2024Open-YOLO 3D: Towards Fast and Accurate Open-Vocabulary 3D Instance Segmentationcited by 30
  6. 2024Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Taskscited by 30
  7. 2024MiKASA: Multi-Key-Anchor & Scene-Aware Transformer for 3D Visual Groundingcited by 29
  8. 2024Can Large Vision-Language Models Correct Semantic Grounding Errors By Themselves?cited by 20
  9. 2025SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Groundingcited by 12
  10. 2024Transcrib3D: 3D Referring Expression Resolution through Large Language Modelscited by 10
  11. 2025ViewSRD: 3D Visual Grounding via Structured Multi-View Decompositioncited by 8
  12. 2024Solving Zero-Shot 3D Visual Grounding as Constraint Satisfaction Problemscited by 7
  13. 2025SeqVLM: Proposal-Guided Multi-View Sequences Reasoning via VLM for Zero-Shot 3D Visual Groundingcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GuideGround: VLM-guided Semantic Understanding and Viewpoint-aware Reasoning for 3D Visual Grounding
  2. 2025Zero-Shot 3D Visual Grounding from Vision-Language Models
  3. 2026UniGround: Universal 3D Visual Grounding via Training-Free Scene Parsing
  4. 2025View-on-Graph: Zero-shot 3D Visual Grounding via Vision-Language Reasoning on Scene Graphs
  5. 2025OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding
  6. 2026TDVR: Joint Text Disambiguation and Viewpoint Reasoning for Zero-Shot 3D Visual Grounding
  7. 2026Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Grounding
  8. 2026Z3D: Zero-Shot 3D Visual Grounding from Images