Towards Physically Executable 3D Gaussian for Embodied Navigation
Not on the map. Why?
Abstract (arXiv)
3D Gaussian Splatting (3DGS), a 3D representation method with photorealistic real-time rendering capabilities, is regarded as an effective tool for narrowing the sim-to-real gap. However, it lacks fine-grained semantics and physical executability for Visual-Language Navigation (VLN). To address this, we propose SAGE-3D (Semantically and Physically Aligned Gaussian Environments for 3D Navigation), a new paradigm that upgrades 3DGS into an executable, semantically and physically aligned environment. It comprises two components: (1) Object-Centric Semantic Grounding, which adds object-level fine-grained annotations to 3DGS; and (2) Physics-Aware Execution Jointing, which embeds collision objects into 3DGS and constructs rich physical interfaces. We release InteriorGS, containing 1K object-annotated 3DGS indoor scene data, and introduce SAGE-Bench, the first 3DGS-based VLN benchmark with 2M VLN data. Experiments show that 3DGS scene data is more difficult to converge, while exhibiting strong generalizability, improving baseline performance by 31% on the VLN-CE Unseen task. Our data and code are available at: https://sage-3d.github.io.
Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 35 of its 37 references that are in the map. First posted on arXiv 24 Oct 2025.
Builds on
35 of 37 references.
References · 35
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 616
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
- 2024gsplat: An Open-Source Library for Gaussian Splattingcited by 285
23 more
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
- 2024A Survey on 3D Gaussian Splattingcited by 224
- 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
- 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
- 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
- 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
- 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
- 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
- 20243DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splattingcited by 88
- 2020Object-and-Action Aware Model for Visual Language Navigationcited by 77
- 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
- 2022Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Searchcited by 67
- 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
- 2023DiffusionLight: Light Probes for Free by Painting a Chrome Ballcited by 64
- 2025VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuningcited by 63
- 2024Robo-GS: A Physics Consistent Spatial-Temporal Model for Robotic Arm with Hybrid Representationcited by 46
- 2024Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Reviewcited by 41
- 2022Iterative Vision-and-Language Navigationcited by 35
- 2024InstanceGaussian: Appearance-Semantic Joint Gaussian Representation for 3D Instance-Level Perceptioncited by 35
- 2023Safe-VLN: Collision Avoidance for Vision-and-Language Navigation of Autonomous Robots Operating in Continuous Environmentscited by 22
- 2024RL-GSBridge: 3D Gaussian Splatting Based Real2Sim2Real Method for Robotic Manipulation Learningcited by 22
- 2025VR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotioncited by 22
- 2025DISCOVERSE: Efficient Robot Simulation in Complex High-Fidelity Environmentscited by 12
Built on by
Citing papers per year: 2025: 1; 2026: 16.
17 papers.
Sort:
By area:
- 2026AIM: Intent-Aware Unified world action Modeling with Spatial Value Mapscited by 12
- 2026Rein3D: Reinforced 3D Indoor Scene Generation with Panoramic Video Diffusion Modelscited by 3
- 2026Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agentscited by 3
- 2026TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Maskingcited by 2
- 2026Visually-grounded Humanoid Agentscited by 2
- 2026POINav: Benchmarking and Enhancing Final-Meters Arrival in Real-World Vision-Language Navigationcited by 2
- 2026Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulationcited by 2
- 2026OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agentscited by 2
- 2026RenderMem: Rendering as Spatial Memory Retrievalcited by 1
- 2026Image2Sim: Scaling Embodied Navigation via Generative Neural Simulatorcited by 1
- 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
- 2026LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigationcited by 1
5 more
- 2026TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Modelcited by 1
- 2025RoboTidy : A 3D Gaussian Splatting Household Tidying Benchmark for Embodied Navigation and Actioncited by 0
- 2026GaussianSelector: Lightweight Human-Guided Object Selection in 3D Gaussian Splatting with Graph Optimizationcited by 0
- 2026UniWAM Technical Report: Unified Mobile Manipulation via Mixed-Stream World-Action Modeling and Manipulation Anchor Pose Supervisioncited by 0
- 2026FireNav: Decision-Oriented Reconstruction Evidence Routing for Vision-Language Navigation in Indoor Fire Environmentscited by 0
- 2026FireNav: Decision-Oriented Reconstruction Evidence Routing for Vision-Language Navigation in Indoor Fire Environments11 Oct 2026
- 2026UniWAM Technical Report: Unified Mobile Manipulation via Mixed-Stream World-Action Modeling and Manipulation Anchor Pose Supervision30 Sep 2026
- 2026OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agents20 Sep 2026
- 2026TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model8 Sep 2026
- 2026LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation31 Aug 2026
- 2026GaussianSelector: Lightweight Human-Guided Object Selection in 3D Gaussian Splatting with Graph Optimization2 Aug 2026
- 2026Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulation29 Jul 2026
- 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation9 Jul 2026
- 2026Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator7 Jul 2026
- 2026POINav: Benchmarking and Enhancing Final-Meters Arrival in Real-World Vision-Language Navigation27 May 2026
- 2026Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agents21 Apr 2026
- 2026AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps13 Apr 2026
5 more
- 2026Rein3D: Reinforced 3D Indoor Scene Generation with Panoramic Video Diffusion Models12 Apr 2026
- 2026Visually-grounded Humanoid Agents9 Apr 2026
- 2026TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking1 Apr 2026
- 2026RenderMem: Rendering as Spatial Memory Retrieval15 Mar 2026
- 2025RoboTidy : A 3D Gaussian Splatting Household Tidying Benchmark for Embodied Navigation and Action18 Nov 2025
- 2026TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Maskingcited by 2
- 2026Visually-grounded Humanoid Agentscited by 2
- 2026POINav: Benchmarking and Enhancing Final-Meters Arrival in Real-World Vision-Language Navigationcited by 2
- 2026Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulationcited by 2
- 2026OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agentscited by 2
- 2026RenderMem: Rendering as Spatial Memory Retrievalcited by 1
- 2026Image2Sim: Scaling Embodied Navigation via Generative Neural Simulatorcited by 1
- 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
- 2026LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigationcited by 1
- 2026TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Modelcited by 1
- 2025RoboTidy : A 3D Gaussian Splatting Household Tidying Benchmark for Embodied Navigation and Actioncited by 0
- 2026GaussianSelector: Lightweight Human-Guided Object Selection in 3D Gaussian Splatting with Graph Optimizationcited by 0
12 most cited of 13 All 13 (JSON)
- 2026FireNav: Decision-Oriented Reconstruction Evidence Routing for Vision-Language Navigation in Indoor Fire Environments11 Oct 2026
- 2026OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agents20 Sep 2026
- 2026TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model8 Sep 2026
- 2026LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation31 Aug 2026
- 2026GaussianSelector: Lightweight Human-Guided Object Selection in 3D Gaussian Splatting with Graph Optimization2 Aug 2026
- 2026Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulation29 Jul 2026
- 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation9 Jul 2026
- 2026Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator7 Jul 2026
- 2026POINav: Benchmarking and Enhancing Final-Meters Arrival in Real-World Vision-Language Navigation27 May 2026
- 2026Visually-grounded Humanoid Agents9 Apr 2026
- 2026TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking1 Apr 2026
- 2026RenderMem: Rendering as Spatial Memory Retrieval15 Mar 2026
12 newest of 13 All 13 (JSON)