mlsanityData to 14 Oct 2026

TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigation

arXiv 2411.16425 ↗First posted 25 Nov 2024cs.CV cs.AI cs.ROcited by 15

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

The Zero-Shot Object Navigation (ZSON) task requires embodied agents to find a previously unseen object by navigating in unfamiliar environments. Such a goal-oriented exploration heavily relies on the ability to perceive, understand, and reason based on the spatial information of the environment. However, current LLM-based approaches convert visual observations to language descriptions and reason in the linguistic space, leading to the loss of spatial information. In this paper, we introduce TopV-Nav, an MLLM-based method that directly reasons on the top-view map with sufficient spatial information. To fully unlock the MLLM's spatial reasoning potential in top-view perspective, we propose the Adaptive Visual Prompt Generation (AVPG) method to adaptively construct semantically-rich top-view map. It enables the agent to directly utilize spatial information contained in the top-view map to conduct thorough reasoning. Besides, we design a Dynamic Map Scaling (DMS) mechanism to dynamically zoom top-view map at preferred scales, enhancing local fine-grained reasoning. Additionally, we devise a Potential Target Driven (PTD) mechanism to predict and to utilize target locations, facilitating global and human-like exploration. Experiments on MP3D and HM3D datasets demonstrate the superiority of our TopV-Nav.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 43 of its 58 references that are in the map. First posted on arXiv 25 Nov 2024.

Builds on

43 of 58 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024TopViewRS: Vision-Language Models as Top-View Spatial Reasoners4 Jun
  2. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs12 Feb
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  4. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models26 May

Other references · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2017Mask R-CNNcited by 7,277
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  5. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  6. 2021Grounded Language-Image Pre-trainingcited by 1,156
  7. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  8. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  9. 2017Cognitive Mapping and Planning for Visual Navigationcited by 212
  10. 2020Neural Topological SLAM for Visual Navigationcited by 195
  11. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  12. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
27 more
  1. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
  2. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 150
  3. 2018Visual Semantic Navigation using Scene Priorscited by 128
  4. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 127
  5. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 117
  6. 2022Habitat-Matterport 3D Semantics Datasetcited by 110
  7. 2018Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learningcited by 93
  8. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 93
  9. 2018Visual Representations for Semantic Target Driven Navigationcited by 89
  10. 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 77
  11. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 65
  12. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  13. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 57
  14. 2024Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Modelscited by 57
  15. 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
  16. 2021Visual Navigation with Spatial Attentioncited by 43
  17. 2020Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentationcited by 40
  18. 2022Reinforced Structured State-Evolution for Vision-Language Navigationcited by 37
  19. 2023iPlanner: Imperative Path Planningcited by 37
  20. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 31
  21. 2024Affordance-Guided Reinforcement Learning via Visual Promptingcited by 23
  22. 2022PEANUT: Predicting and Navigating to Unseen Targetscited by 22
  23. 2022Navigating to Objects in Unseen Environments by Distance Predictioncited by 16
  24. 2023ESceme: Vision-and-Language Navigation with Episodic Scene Memorycited by 10
  25. 2023MTG: Mapless Trajectory Generator with Traversability Coverage for Outdoor Navigationcited by 10
  26. 2021Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graphcited by 7
  27. 2018Learning and Planning with a Semantic Modelcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 8.

15 papers.

Sort:

By area:

  1. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 43
  2. 2025OctoNav: Towards Generalist Embodied Navigationcited by 34
  3. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  4. 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Sciencecited by 12
  5. 2025DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigationcited by 6
  6. 2026FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigationcited by 4
  7. 2025Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neurosciencecited by 3
  8. 2025Towards Physics-informed Spatial Intelligence with Human Priors: An Autonomous Driving Pilot Studycited by 2
  9. 2026MerNav: A Highly Generalizable Memory-Execute-Review Framework for Zero-Shot Object Goal Navigationcited by 2
  10. 2026Parse, Search, and Confirmation: Training-Free Aerial Vision-and-Dialog Navigation with Chain-of-Thought Reasoning and Structured Spatial Memorycited by 1
  11. 2026Design-MLLM: A Reinforcement Alignment Framework for Verifiable and Aesthetic Interior Designcited by 0
  12. 2026SR-Nav: Spatial Relationships Matter for Zero-shot Object Goal Navigationcited by 0
3 more
  1. 2026HiRO-Nav: Hybrid ReasOning Enables Efficient Embodied Navigationcited by 0
  2. 2026Multi-Scale Gaussian-Language Map for Zero-shot Embodied Navigation and Reasoningcited by 0
  3. 2026NavOne: One-Step Global Planning for Vision-Language Navigation on Top-Down Mapscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Think, Remember, Navigate: Zero-Shot Object-Goal Navigation with VLM-Powered Reasoning
  2. 2025BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation
  3. 2025PIGEON: VLM-Driven Object Navigation via Points of Interest Selection
  4. 20263DGSNav: Enhancing Vision-Language Model Reasoning for Object Navigation via Active 3D Gaussian Splatting
  5. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMs
  6. 2026Hydra-Nav: Object Navigation via Adaptive Dual-Process Reasoning
  7. 2024Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigation
  8. 2024End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering