mlsanityData to 13 Oct 2026

Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skill

arXiv 2309.10309 ↗First posted 19 Sep 2023cs.ROcited by 75

Not on the map. Why?

Abstract (arXiv)

Zero-shot object navigation is a challenging task for home-assistance robots. This task emphasizes visual grounding, commonsense inference and locomotion abilities, where the first two are inherent in foundation models. But for the locomotion part, most works still depend on map-based planning approaches. The gap between RGB space and map space makes it difficult to directly transfer the knowledge from foundation models to navigation tasks. In this work, we propose a Pixel-guided Navigation skill (PixNav), which bridges the gap between the foundation models and the embodied navigation task. It is straightforward for recent foundation models to indicate an object by pixels, and with pixels as the goal specification, our method becomes a versatile navigation policy towards all different kinds of objects. Besides, our PixNav is a pure RGB-based policy that can reduce the cost of home-assistance robots. Experiments demonstrate the robustness of the PixNav which achieves 80+% success rate in the local path-planning task. To perform long-horizon object navigation, we design an LLM-based planner to utilize the commonsense knowledge between objects and rooms to select the best waypoint. Evaluations across both photorealistic indoor simulators and real-world environments validate the effectiveness of our proposed navigation strategy. Code and video demos are available at https://github.com/wzcai99/Pixel-Navigator.

Key facts. Cited by 75 papers in the map (data through 13 Oct 2026). First posted on arXiv 19 Sep 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2023: 1; 2024: 22; 2025: 34; 2026: 18.

75 papers.

Sort:

By area:

  1. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 169
  2. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 152
  3. 2023Large Language Models for Robotics: A Surveycited by 132
  4. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 111
  5. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 72
  6. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Modelscited by 66
  7. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 62
  8. 2024RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesiscited by 56
  9. 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 56
  10. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
  11. 2025NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidancecited by 46
  12. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 42
38 more
  1. 2024CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMscited by 32
  2. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
  3. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  4. 2024An Interactive Agent Foundation Modelcited by 25
  5. 2024ManipVQA: Injecting Robotic Affordance and Physically Grounded Information into Multi-Modal Large Language Modelscited by 25
  6. 2024MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chainscited by 25
  7. 2025RefDrone: A Challenging Benchmark for Referring Expression Comprehension in Drone Scenescited by 17
  8. 2024TopV-Nav: Unlocking the Top-View Spatial Reasoning Potential of MLLM for Zero-shot Object Navigationcited by 15
  9. 2024Enhancing Multi-Robot Semantic Navigation Through Multimodal Chain-of-Thought Score Collaborationcited by 14
  10. 2025NavigateDiff: Visual Predictors are Zero-Shot Navigation Assistantscited by 11
  11. 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10
  12. 2025PanoNav: Mapless Zero-Shot Object Navigation with Panoramic Scene Parsing and Dynamic Memorycited by 9
  13. 2024CAMON: Cooperative Agents for Multi-Object Navigation with LLM-based Conversationscited by 7
  14. 2025FlySearch: Exploring how vision-language models explorecited by 7
  15. 2025CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulationcited by 7
  16. 2025CogDDN: A Cognitive Demand-Driven Navigation with Decision Optimization and Dual-Process Thinkingcited by 7
  17. 2025From reactive to cognitive: brain-inspired spatial intelligence for embodied agentscited by 7
  18. 2025TANGO: Traversability-Aware Navigation with Local Metric Control for Topological Goalscited by 7
  19. 2024UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Modelscited by 6
  20. 2025DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigationcited by 6
  21. 2025UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectoriescited by 6
  22. 2024DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenescited by 5
  23. 2025ObjectReact: Learning Object-Relative Control for Visual Navigationcited by 5
  24. 2025EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrievalcited by 5
  25. 2026LongNav-R1: Horizon-Adaptive Multi-Turn RL for Long-Horizon VLA Navigationcited by 5
  26. 2024Find Everything: A General Vision Language Model Approach to Multi-Object Searchcited by 4
  27. 2025RATE-Nav: Region-Aware Termination Enhancement for Zero-shot Object Navigation with Vision-Language Modelscited by 4
  28. 2025Weakly-supervised VLM-guided Partial Contrastive Learning for Visual Language Navigationcited by 4
  29. 2025History-Augmented Vision-Language Models for Frontier-Based Zero-Shot Object Navigationcited by 4
  30. 2025AgriVLN: Vision-and-Language Navigation for Agricultural Robotscited by 4
  31. 2025Human-like Navigation in a World Built for Humanscited by 4
  32. 2024Cog-GA: A Large Language Models-based Generative Agent for Vision-Language Navigation in Continuous Environmentscited by 3
  33. 2025Distilling LLM Prior to Flow Model for Generalizable Agent's Imagination in Object Goal Navigationcited by 3
  34. 2025MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learningcited by 3
  35. 2024Position Paper: Agent AI Towards a Holistic Intelligencecited by 2
  36. 2025EMMOE: A Comprehensive Benchmark for Embodied Mobile Manipulation in Open Environmentscited by 2
  37. 2025RANa: Retrieval-Augmented Navigationcited by 2
  38. 2026MerNav: A Highly Generalizable Memory-Execute-Review Framework for Zero-Shot Object Goal Navigationcited by 2

50 most cited of 75 All 75 (JSON)