Hierarchical 3D Scene Graph Construction and Belief-based Planning for Semantic Navigation
Not on the map. Why?
Abstract (arXiv)
Semantic navigation is a fundamental task for embodied agents operating in unseen environments, requiring both semantic understanding and long-term decision-making. Recent foundation models have empowered agents with rich semantic priors for this task. However, without structured global representations, decision-making often falls back on local observations and greedy strategies, resulting in inefficient exploration and myopic behaviors, especially in long-distance navigation. To address these challenges, we propose a zero-shot semantic navigation framework. Our method incrementally maintains an online Hierarchical 3D Scene Graph (HSG) to form a multi-granular semantic topology over objects, zones, and regions, serving as a compact state abstraction for global planning. Building on this memory, we introduce a hierarchical belief-based planning framework that fuses semantic priors with exploration evidence on the HSG, and performs finite-horizon rollouts on an HSG-based simulator to explicitly estimate the long-term expected returns of candidate macro-actions. This enables globally consistent decisions and reduces redundant backtracking. Extensive experiments in high-fidelity simulation environments across multiple tasks and datasets demonstrate that our method outperforms existing state-of-the-art methods, particularly in long-distance scenarios, where our approach improves SR and SPL by an average of 9.4% and 5.0%, respectively.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 30 Jun 2026.
Builds on
34 of 43 references.
References · 34
- 2025Qwen3 Technical Reportcited by 8,195
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2018On Evaluation of Embodied Navigation Agentscited by 526
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
- 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
- 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 280
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
- 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 195
- 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
22 more
- 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
- 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 162
- 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
- 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 146
- 2021Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphscited by 144
- 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
- 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 118
- 2022Habitat-Matterport 3D Semantics Datasetcited by 109
- 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
- 2023GOAT: GO to Any Thingcited by 89
- 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 76
- 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 66
- 2023Foundations of Spatial Perception for Robotics: Hierarchical Representations and Real-time Systemscited by 60
- 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 56
- 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
- 2020FUEL: Fast UAV Exploration using Incremental Frontier Structure and Hierarchical Planningcited by 42
- 2024ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imaginationcited by 38
- 2024CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMscited by 32
- 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
- 2025ApexNav: An Adaptive Exploration Strategy for Zero-Shot Object Navigation with Target-centric Semantic Fusioncited by 29
- 2025BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigationcited by 18
- 2026VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memorycited by 10
Built on by
Not cited yet.