3DGSNav: Enhancing Vision-Language Model Reasoning for Object Navigation via Active 3D Gaussian Splatting
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Object navigation is a core capability of embodied intelligence, enabling an agent to locate target objects in unknown environments. Recent advances in vision-language models (VLMs) have facilitated zero-shot object navigation (ZSON). However, existing methods often rely on scene abstractions that convert environments into semantic maps or textual representations, causing high-level decision making to be constrained by the accuracy of low-level perception. In this work, we present 3DGSNav, a novel ZSON framework that embeds 3D Gaussian Splatting (3DGS) as persistent memory for VLMs to enhance spatial reasoning. Through active perception, 3DGSNav incrementally constructs a 3DGS representation of the environment, enabling trajectory-guided free-viewpoint rendering of frontier-aware first-person views. Moreover, we design structured visual prompts and integrate them with Chain-of-Thought (CoT) prompting to further improve VLM reasoning. During navigation, a real-time object detector filters potential targets, while VLM-driven active viewpoint switching performs target re-verification, ensuring efficient and reliable recognition. Extensive evaluations across multiple benchmarks and real-world experiments on a quadruped robot demonstrate that our method achieves robust and competitive performance against state-of-the-art approaches.The Project Page:https://aczheng-cai.github.io/3dgsnav.github.io/
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 42 references that are in the map. First posted on arXiv 12 Feb 2026.
Builds on
36 of 42 references.
Closest ancestors · 5
Same direction, newest first
- 2025STRIVE: Structured Representation Integrating VLM Reasoning for Efficient Object Navigation10 May
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
- 2024Zero-shot Object Navigation with Vision-Language Models Reasoning24 Oct
- 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov
- 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models26 May
Other references · 31
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
- 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
- 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
- 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 154
19 more
- 2024Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understandingcited by 131
- 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 118
- 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
- 2022Habitat-Matterport 3D Semantics Datasetcited by 109
- 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 91
- 2018Visual Representations for Semantic Target Driven Navigationcited by 89
- 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 73
- 20253D-LLaVA: Towards Generalist 3D LMMs with Omni Superpoint Transformercited by 67
- 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 66
- 2025Ross3D: Reconstructive Visual Instruction Tuning with 3D-Awarenesscited by 61
- 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 60
- 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontierscited by 53
- 2025Inst3D-LMM: Instance-Aware 3D Scene Understanding with Multi-modal Instruction Tuningcited by 50
- 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
- 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
- 2025ApexNav: An Adaptive Exploration Strategy for Zero-Shot Object Navigation with Target-centric Semantic Fusioncited by 29
- 2024TriHelper: Zero-Shot Object Navigation with Dynamic Assistancecited by 24
- 2022Search for or Navigate to? Dual Adaptive Thinking for Object Navigationcited by 19
- 2025BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigationcited by 18
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026AERR-Nav: Adaptive Exploration-Recovery-Reminiscing Strategy for Zero-Shot Object Navigationcited by 1
- 2026CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperationcited by 0
- 2026Affordance-Conditioned Decision Making: Bridging the Semantic-Spatial Gap in Zero-Shot Cross-Floor Vision-and-Language Navigationcited by 0
- 2026Affordance-Conditioned Decision Making: Bridging the Semantic-Spatial Gap in Zero-Shot Cross-Floor Vision-and-Language Navigation26 Sep 2026
- 2026CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation25 Sep 2026
- 2026AERR-Nav: Adaptive Exploration-Recovery-Reminiscing Strategy for Zero-Shot Object Navigation18 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026GSMem: 3D Gaussian Splatting as Persistent Spatial Memory for Zero-Shot Embodied Exploration and Reasoning
- 2026SplatReasoner: Enhancing Embodied Reasoning and Grounding by Novel View Synthesis
- 2026GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigation
- 2026SEDualVLN: A Spatially-Enhanced Dual-System for Vision-Language Navigation
- 2025GaussianVLM: Scene-centric 3D Vision-Language Models using Language-aligned Gaussian Splats for Embodied Reasoning and Beyond
- 2026One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints
- 2025History-Augmented Vision-Language Models for Frontier-Based Zero-Shot Object Navigation
- 2026OmniVLN: Omnidirectional 3D Perception and Token-Efficient LLM Reasoning for Visual-Language Navigation across Air and Ground Platforms