# ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning

arXiv: https://arxiv.org/abs/2503.23297 · HTML: https://mlsanity.com/paper/2503.23297/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 6 of its 40 references that are in the map.
- First posted on arXiv 30 Mar 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 6 of 6)

- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [BlenderProc](https://mlsanity.com/paper/1911.01911/) (2019)
- [Multi-CLIP: Contrastive Vision-Language Pre-training for Question Answering tasks in 3D Scenes](https://mlsanity.com/paper/2306.02329/) (2023)
- [Reasoning3D -- Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models](https://mlsanity.com/paper/2405.19326/) (2024)
- [In-Place Panoptic Radiance Field Segmentation with Perceptual Prior for 3D Scene Understanding](https://mlsanity.com/paper/2410.04529/) (2024)

## Built on by (most cited first, 25 of 25)

- [How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective](https://mlsanity.com/paper/2509.18905/) (2025)
- [Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks](https://mlsanity.com/paper/2510.25760/) (2025)
- [N3D-VLM: Native 3D Grounding Enables Accurate Spatial Reasoning in Vision-Language Models](https://mlsanity.com/paper/2512.16561/) (2025)
- [A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation](https://mlsanity.com/paper/2508.09977/) (2025)
- [ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulation](https://mlsanity.com/paper/2601.08325/) (2026)
- [Spatial-Temporal Aware Visuomotor Diffusion Policy Learning](https://mlsanity.com/paper/2507.06710/) (2025)
- [AnchorSplat: Feed-Forward 3D Gaussian Splatting with 3D Geometric Priors](https://mlsanity.com/paper/2604.07053/) (2026)
- [A Neural Representation Framework with LLM-Driven Spatial Reasoning for Open-Vocabulary 3D Visual Grounding](https://mlsanity.com/paper/2507.06719/) (2025)
- [REALM: An MLLM-Agent Framework for Open World 3D Reasoning Segmentation and Editing on Gaussian Splatting](https://mlsanity.com/paper/2510.16410/) (2025)
- [Zero-Shot Visual Grounding in 3D Gaussians via View Retrieval](https://mlsanity.com/paper/2509.15871/) (2025)
- [From 2D to 3D Cognition: A Brief Survey of General World Models](https://mlsanity.com/paper/2506.20134/) (2025)
- [SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs](https://mlsanity.com/paper/2602.22716/) (2026)
- [WeaveLA: Event Driven Cross-Subtask Latent Memory Weaving for Repetitive Robot Manipulation](https://mlsanity.com/paper/2606.17463/) (2026)
- [Scenes as Tokens: Multi-Scale Normal Distributions Transform Tokenizer for General 3D Vision-Language Understanding](https://mlsanity.com/paper/2511.21191/) (2025)
- [OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding](https://mlsanity.com/paper/2512.23020/) (2025)
- [G2P: Gaussian-to-Point Attribute Alignment for Boundary-Aware 3D Segmentation](https://mlsanity.com/paper/2601.03510/) (2026)
- [SpatialPoint: Spatial-aware Point Prediction for Embodied Localization](https://mlsanity.com/paper/2603.26690/) (2026)
- [VADF: Vision-Adaptive Diffusion Policy Framework for Efficient Robotic Manipulation](https://mlsanity.com/paper/2604.15938/) (2026)
- [PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding](https://mlsanity.com/paper/2606.31148/) (2026)
- [Seek to Segment: Active Perception for Panoramic Referring Segmentation](https://mlsanity.com/paper/2607.02497/) (2026)
- [CapFrame: Text-Instructed Viewpoint Grounding in 3D Gaussian Scenes via Geometric Pseudo Labels](https://mlsanity.com/paper/2608.30342/) (2026)
- [CitySTAR: Structured and Topology-Aware Reasoning for Open-Vocabulary Urban 3D Grounding](https://mlsanity.com/paper/2609.19911/) (2026)
- [CoRef-GS: Cooperative Referring Gaussian Splatting for Multi-Agent Scene Understanding](https://mlsanity.com/paper/2609.20586/) (2026)
- [EviSplat: Preserving Multi-View Evidence in 3D Gaussian Splatting for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2609.34853/) (2026)
- [S4VY: Segment Anything in Feed-Forward 4D Visual Geometry](https://mlsanity.com/paper/2609.36875/) (2026)

## Abstract (arXiv)

Open-vocabulary 3D visual grounding and reasoning aim to localize objects in a scene based on implicit language descriptions, even when they are occluded. This ability is crucial for tasks such as vision-language navigation and autonomous robotics. However, current methods struggle because they rely heavily on fine-tuning with 3D annotations and mask proposals, which limits their ability to handle diverse semantics and common knowledge required for effective reasoning. In this work, we propose ReasonGrounder, an LVLM-guided framework that uses hierarchical 3D feature Gaussian fields for adaptive grouping based on physical scale, enabling open-vocabulary 3D grounding and reasoning. ReasonGrounder interprets implicit instructions using large vision-language models (LVLM) and localizes occluded objects through 3D Gaussian splatting. By incorporating 2D segmentation masks from the SAM and multi-view CLIP embeddings, ReasonGrounder selects Gaussian groups based on object scale, enabling accurate localization through both explicit and implicit language understanding, even in novel, occluded views. We also contribute ReasoningGD, a new dataset containing over 10K scenes and 2 million annotations for evaluating open-vocabulary 3D grounding and amodal perception under occlusion. Experiments show that ReasonGrounder significantly improves 3D grounding accuracy in real-world scenarios.