Hierarchical Object-to-Zone Graph for Object Navigation
Not on the map. Why?
Abstract (arXiv)
The goal of object navigation is to reach the expected objects according to visual information in the unseen environments. Previous works usually implement deep models to train an agent to predict actions in real-time. However, in the unseen environment, when the target object is not in egocentric view, the agent may not be able to make wise decisions due to the lack of guidance. In this paper, we propose a hierarchical object-to-zone (HOZ) graph to guide the agent in a coarse-to-fine manner, and an online-learning mechanism is also proposed to update HOZ according to the real-time observation in new environments. In particular, the HOZ graph is composed of scene nodes, zone nodes and object nodes. With the pre-learned HOZ graph, the real-time observation and the target goal, the agent can constantly plan an optimal path from zone to zone. In the estimated path, the next potential zone is regarded as sub-goal, which is also fed into the deep reinforcement learning model for action prediction. Our methods are evaluated on the AI2-Thor simulator. In addition to widely used evaluation metrics SR and SPL, we also propose a new evaluation metric of SAE that focuses on the effective action rate. Experimental results demonstrate the effectiveness and efficiency of our proposed method.
Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 34 of its 45 references that are in the map. First posted on arXiv 5 Sep 2021.
Builds on
34 of 45 references.
References · 34
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
- 2018On Evaluation of Embodied Navigation Agentscited by 526
- 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
- 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
- 2020Learning to Explore using Active Neural SLAMcited by 308
22 more
- 2016CAD2RL: Real Single-Image Flight without a Single Real Imagecited by 227
- 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
- 2018Semi-parametric Topological Memory for Navigationcited by 203
- 2020Neural Topological SLAM for Visual Navigationcited by 193
- 2016Learning to Navigate in Complex Environmentscited by 192
- 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
- 2018Zero-shot Recognition via Semantic Embeddings and Knowledge Graphscited by 155
- 2018Visual Semantic Navigation using Scene Priorscited by 127
- 2019Learning Exploration Policies for Navigationcited by 113
- 2019Scene Memory Transformer for Embodied Agents in Long-Horizon Taskscited by 106
- 2018Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learningcited by 91
- 2018Visual Representations for Semantic Target Driven Navigationcited by 89
- 2017Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigationcited by 74
- 2016The More You Know: Using Knowledge Graphs for Image Classificationcited by 63
- 2020Learning Object Relation Graph and Tentative Policy for Visual Navigationcited by 62
- 2019Combining Optimal Control and Learning for Visual Navigation in Novel Environmentscited by 50
- 2019Bayesian Relational Memory for Semantic Visual Navigationcited by 48
- 2019A Behavioral Approach to Visual Navigation with Graph Localization Networkscited by 40
- 2019Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigationcited by 39
- 2019Scaling Local Control to Large-Scale Topological Navigationcited by 37
- 2015segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detectioncited by 7
- 2015Learning Contextual Dependencies with Convolutional Hierarchical Recurrent Neural Networkscited by 4
Built on by
Citing papers per year: 2021: 1; 2022: 5; 2023: 9; 2024: 11; 2025: 3; 2026: 5.
34 papers.
Sort:
By area:
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
- 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
- 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
- 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
- 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 63
- 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontierscited by 53
- 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
- 2022Search for or Navigate to? Dual Adaptive Thinking for Object Navigationcited by 19
- 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18
- 2022Unbiased Directed Object Attention Graph for Object Navigationcited by 17
- 2024Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigationcited by 15
- 2023Graph based Environment Representation for Vision-and-Language Navigation in Continuous Environmentscited by 9
22 more
- 2025ForesightNav: Learning Scene Imagination for Efficient Explorationcited by 9
- 2023Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigationcited by 8
- 2024Aligning Knowledge Graph with Visual Perception for Object-goal Navigationcited by 8
- 2025EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrievalcited by 5
- 2022Spatial Relation Graph and Graph Convolutional Network for Object Goal Navigationcited by 3
- 2023Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidancecited by 3
- 2023Building Category Graphs Representation with Spatial and Temporal Attention for Visual Navigationcited by 2
- 2026PSG-Nav: Probabilistic Scene Graph Navigation via Multiverse Decision Makingcited by 2
- 2022Unsupervised Visual Odometry and Action Integration for PointGoal Navigation in Indoor Environmentcited by 1
- 2023Task-Driven Graph Attention for Hierarchical Relational Object Navigationcited by 1
- 2023Learning to Terminate in Object Navigationcited by 1
- 2024RobKiNet: Robotic Kinematics Informed Neural Network for Optimal Robot Configuration Predictioncited by 1
- 2021Symmetry-aware Neural Architecture for Embodied Visual Navigationcited by 0
- 2024Temporal-Spatial Object Relations Modeling for Vision-and-Language Navigationcited by 0
- 2024Leveraging Unknown Objects to Construct Labeled-Unlabeled Meta-Relationships for Zero-Shot Object Navigationcited by 0
- 2024Efficient Strategy Learning by Decoupling Searching and Pathfinding for Object Navigationcited by 0
- 2024Semantic Environment Atlas for Object-Goal Navigationcited by 0
- 2025Collision-Aware Object-Goal Visual Navigation via Two-Stage Deep Reinforcement Learningcited by 0
- 2026CoReLIN: Constraint-based Reasoning for Zero-shot Lifelong Interactive Navigationcited by 0
- 2026SR-Nav: Spatial Relationships Matter for Zero-shot Object Goal Navigationcited by 0
- 2026GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigationcited by 0
- 2026Graph-MambaNav: Spatial-Temporal Graph Mamba Leveraging Object-Relation Knowledge for Object-Goal Navigationcited by 0
- 2026Graph-MambaNav: Spatial-Temporal Graph Mamba Leveraging Object-Relation Knowledge for Object-Goal Navigation13 Aug 2026
- 2026PSG-Nav: Probabilistic Scene Graph Navigation via Multiverse Decision Making31 May 2026
- 2026GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation21 May 2026
- 2026SR-Nav: Spatial Relationships Matter for Zero-shot Object Goal Navigation19 Mar 2026
- 2026CoReLIN: Constraint-based Reasoning for Zero-shot Lifelong Interactive Navigation23 Feb 2026
- 2025EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrieval21 Oct 2025
- 2025ForesightNav: Learning Scene Imagination for Efficient Exploration22 Apr 2025
- 2025Collision-Aware Object-Goal Visual Navigation via Two-Stage Deep Reinforcement Learning19 Feb 2025
- 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Tasks26 Nov 2024
- 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation10 Oct 2024
- 2024Semantic Environment Atlas for Object-Goal Navigation5 Oct 2024
- 2024Efficient Strategy Learning by Decoupling Searching and Pathfinding for Object Navigation20 Jun 2024
22 more
- 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigation14 Jun 2024
- 2024Leveraging Unknown Objects to Construct Labeled-Unlabeled Meta-Relationships for Zero-Shot Object Navigation24 May 2024
- 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation2 Apr 2024
- 2024Temporal-Spatial Object Relations Modeling for Vision-and-Language Navigation23 Mar 2024
- 2024Aligning Knowledge Graph with Visual Perception for Object-goal Navigation29 Feb 2024
- 2024RobKiNet: Robotic Kinematics Informed Neural Network for Optimal Robot Configuration Prediction26 Feb 2024
- 2024Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigation25 Feb 2024
- 2023Building Category Graphs Representation with Spatial and Temporal Attention for Visual Navigation6 Dec 2023
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation6 Dec 2023
- 2023Learning to Terminate in Object Navigation28 Sep 2023
- 2023Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidance24 Aug 2023
- 2023GridMM: Grid Memory Map for Vision-and-Language Navigation24 Jul 2023
- 2023Task-Driven Graph Attention for Hierarchical Relational Object Navigation23 Jun 2023
- 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers26 May 2023
- 2023Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigation3 Feb 2023
- 2023Graph based Environment Representation for Vision-and-Language Navigation in Continuous Environments11 Jan 2023
- 2022Unsupervised Visual Odometry and Action Integration for PointGoal Navigation in Indoor Environment2 Oct 2022
- 2022Spatial Relation Graph and Graph Convolutional Network for Object Goal Navigation27 Aug 2022
- 2022Search for or Navigate to? Dual Adaptive Thinking for Object Navigation1 Aug 2022
- 2022Unbiased Directed Object Attention Graph for Object Navigation9 Apr 2022
- 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning25 Jan 2022
- 2021Symmetry-aware Neural Architecture for Embodied Visual Navigation17 Dec 2021
- 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
- 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
- 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 106
- 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 63
- 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontierscited by 53
- 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
- 2022Search for or Navigate to? Dual Adaptive Thinking for Object Navigationcited by 19
- 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18
- 2022Unbiased Directed Object Attention Graph for Object Navigationcited by 17
- 2024Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigationcited by 15
- 2023Graph based Environment Representation for Vision-and-Language Navigation in Continuous Environmentscited by 9
- 2025ForesightNav: Learning Scene Imagination for Efficient Explorationcited by 9
12 most cited of 30 All 30 (JSON)
- 2026PSG-Nav: Probabilistic Scene Graph Navigation via Multiverse Decision Making31 May 2026
- 2026SR-Nav: Spatial Relationships Matter for Zero-shot Object Goal Navigation19 Mar 2026
- 2025EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrieval21 Oct 2025
- 2025ForesightNav: Learning Scene Imagination for Efficient Exploration22 Apr 2025
- 2025Collision-Aware Object-Goal Visual Navigation via Two-Stage Deep Reinforcement Learning19 Feb 2025
- 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Tasks26 Nov 2024
- 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigation10 Oct 2024
- 2024Semantic Environment Atlas for Object-Goal Navigation5 Oct 2024
- 2024Efficient Strategy Learning by Decoupling Searching and Pathfinding for Object Navigation20 Jun 2024
- 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigation14 Jun 2024
- 2024Leveraging Unknown Objects to Construct Labeled-Unlabeled Meta-Relationships for Zero-Shot Object Navigation24 May 2024
- 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation2 Apr 2024
12 newest of 30 All 30 (JSON)