# Towards Physically Executable 3D Gaussian for Embodied Navigation

arXiv: https://arxiv.org/abs/2510.21307 · HTML: https://mlsanity.com/paper/2510.21307/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 37 references that are in the map.
- First posted on arXiv 24 Oct 2025.

## Builds on (most cited first, 30 of 35)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering](https://mlsanity.com/paper/2311.12775/) (2023)
- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2004.02857/) (2020)
- [Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding](https://mlsanity.com/paper/2010.07954/) (2020)
- [gsplat: An Open-Source Library for Gaussian Splatting](https://mlsanity.com/paper/2409.06765/) (2024)
- [NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation](https://mlsanity.com/paper/2402.15852/) (2024)
- [A Survey on 3D Gaussian Splatting](https://mlsanity.com/paper/2401.03890/) (2024)
- [NaVILA: Legged Robot Vision-Language-Action Model for Navigation](https://mlsanity.com/paper/2412.04453/) (2024)
- [Towards Learning a Generalist Model for Embodied Navigation](https://mlsanity.com/paper/2312.02010/) (2023)
- [StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling](https://mlsanity.com/paper/2507.05240/) (2025)
- [Scaling Data Generation in Vision-and-Language Navigation](https://mlsanity.com/paper/2307.15644/) (2023)
- [NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models](https://mlsanity.com/paper/2407.12366/) (2024)
- [GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation](https://mlsanity.com/paper/2404.06609/) (2024)
- [3DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splatting](https://mlsanity.com/paper/2412.12507/) (2024)
- [Object-and-Action Aware Model for Visual Language Navigation](https://mlsanity.com/paper/2007.14626/) (2020)
- [HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigation](https://mlsanity.com/paper/2409.14296/) (2024)
- [Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Search](https://mlsanity.com/paper/2205.02961/) (2022)
- [Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Method](https://mlsanity.com/paper/2412.09082/) (2024)
- [DiffusionLight: Light Probes for Free by Painting a Chrome Ball](https://mlsanity.com/paper/2312.09168/) (2023)
- [VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.17221/) (2025)
- [Robo-GS: A Physics Consistent Spatial-Temporal Model for Robotic Arm with Hybrid Representation](https://mlsanity.com/paper/2408.14873/) (2024)
- [Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Review](https://mlsanity.com/paper/2405.03417/) (2024)
- [Iterative Vision-and-Language Navigation](https://mlsanity.com/paper/2210.03087/) (2022)

## Built on by (most cited first, 17 of 17)

- [AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps](https://mlsanity.com/paper/2604.11135/) (2026)
- [Rein3D: Reinforced 3D Indoor Scene Generation with Panoramic Video Diffusion Models](https://mlsanity.com/paper/2604.10578/) (2026)
- [Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agents](https://mlsanity.com/paper/2604.19034/) (2026)
- [TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking](https://mlsanity.com/paper/2604.01207/) (2026)
- [Visually-grounded Humanoid Agents](https://mlsanity.com/paper/2604.08509/) (2026)
- [POINav: Benchmarking and Enhancing Final-Meters Arrival in Real-World Vision-Language Navigation](https://mlsanity.com/paper/2605.28237/) (2026)
- [Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulation](https://mlsanity.com/paper/2607.26646/) (2026)
- [OmniEcho: Audio-Visual Spatial Understanding for Omni-Modal Embodied Agents](https://mlsanity.com/paper/2609.23407/) (2026)
- [RenderMem: Rendering as Spatial Memory Retrieval](https://mlsanity.com/paper/2603.14669/) (2026)
- [Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator](https://mlsanity.com/paper/2607.05765/) (2026)
- [A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation](https://mlsanity.com/paper/2607.09792/) (2026)
- [LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation](https://mlsanity.com/paper/2608.30935/) (2026)
- [TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model](https://mlsanity.com/paper/2609.09158/) (2026)
- [RoboTidy : A 3D Gaussian Splatting Household Tidying Benchmark for Embodied Navigation and Action](https://mlsanity.com/paper/2511.14161/) (2025)
- [GaussianSelector: Lightweight Human-Guided Object Selection in 3D Gaussian Splatting with Graph Optimization](https://mlsanity.com/paper/2608.01492/) (2026)
- [UniWAM Technical Report: Unified Mobile Manipulation via Mixed-Stream World-Action Modeling and Manipulation Anchor Pose Supervision](https://mlsanity.com/paper/2609.39388/) (2026)
- [FireNav: Decision-Oriented Reconstruction Evidence Routing for Vision-Language Navigation in Indoor Fire Environments](https://mlsanity.com/paper/2610.15595/) (2026)

## Abstract (arXiv)

3D Gaussian Splatting (3DGS), a 3D representation method with photorealistic real-time rendering capabilities, is regarded as an effective tool for narrowing the sim-to-real gap. However, it lacks fine-grained semantics and physical executability for Visual-Language Navigation (VLN). To address this, we propose SAGE-3D (Semantically and Physically Aligned Gaussian Environments for 3D Navigation), a new paradigm that upgrades 3DGS into an executable, semantically and physically aligned environment. It comprises two components: (1) Object-Centric Semantic Grounding, which adds object-level fine-grained annotations to 3DGS; and (2) Physics-Aware Execution Jointing, which embeds collision objects into 3DGS and constructs rich physical interfaces. We release InteriorGS, containing 1K object-annotated 3DGS indoor scene data, and introduce SAGE-Bench, the first 3DGS-based VLN benchmark with 2M VLN data. Experiments show that 3DGS scene data is more difficult to converge, while exhibiting strong generalizability, improving baseline performance by 31% on the VLN-CE Unseen task. Our data and code are available at: https://sage-3d.github.io.