Habitat-GS: A High-Fidelity Navigation Simulator with Dynamic Gaussian Splatting
Not on the map. Why?
Abstract (arXiv)
Training embodied AI agents depends critically on the visual fidelity of simulation environments and the ability to model dynamic humans. Current simulators predominantly rely on mesh-based rasterization, for which photorealistic assets are costly to author at scale, and their support for dynamic human avatars is largely constrained to mesh representations, hindering agent generalization to human-populated real-world scenarios. We present Habitat-GS, a navigation-centric embodied AI simulator extended from Habitat-Sim that integrates 3D Gaussian Splatting scene rendering and drivable gaussian avatars while maintaining full compatibility with the Habitat ecosystem. Our system implements a 3DGS renderer for real-time photorealistic rendering and supports scalable 3DGS asset import from diverse sources. For dynamic human modeling, we introduce a gaussian avatar module that enables each avatar to simultaneously serve as a photorealistic visual entity and an effective navigation obstacle, allowing agents to learn human-aware behaviors in realistic settings. Experiments on point-goal navigation demonstrate that agents trained on 3DGS scenes achieve stronger cross-domain generalization. Evaluations on avatar-aware navigation further confirm that gaussian avatars enable effective human-aware navigation, while performance benchmarks validate the system's scalability. Code is available at https://github.com/zju3dv/habitat-gs.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 14 Apr 2026.
Builds on
29 of 42 references.
References · 29
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
- 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
17 more
- 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 737
- 2023Mip-Splatting: Alias-free 3D Gaussian Splattingcited by 667
- 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2018On Evaluation of Embodied Navigation Agentscited by 547
- 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 362
- 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 310
- 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 250
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
- 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 194
- 2023GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussianscited by 191
- 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 155
- 2021The Wanderings of Odysseus in 3D Scenescited by 59
- 2024InCrowd-VI: A Realistic Visual-Inertial Dataset for Evaluating SLAM in Indoor Pedestrian-Rich Spaces for Human Navigationcited by 3
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation Systemcited by 11
- 2026Surflo: Consistent 3D Surface Flow Model with Global Statecited by 1
- 2026LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigationcited by 1
- 2026OptiSight: Bridging Semantic Reasoning and Geometric Control for Embodied Navigationcited by 0
- 2026LightBridge: Feed-Forward Generative Relighting for 3D Gaussian Splattingcited by 0
- 2026LightBridge: Feed-Forward Generative Relighting for 3D Gaussian Splatting2 Sep 2026
- 2026LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation31 Aug 2026
- 2026OptiSight: Bridging Semantic Reasoning and Geometric Control for Embodied Navigation24 Aug 2026
- 2026Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation System16 Jun 2026
- 2026Surflo: Consistent 3D Surface Flow Model with Global State11 Jun 2026