mlsanityData to 7 Oct 2026

Navigating to Objects Specified by Images

arXiv 2304.01192 ↗First posted 3 Apr 2023cs.CV cs.ROcited by 50

Not on the map. Why?

Abstract (arXiv)

Images are a convenient way to specify which particular object instance an embodied agent should navigate to. Solving this task requires semantic visual reasoning and exploration of unknown environments. We present a system that can perform this task in both simulation and the real world. Our modular method solves sub-tasks of exploration, goal instance re-identification, goal localization, and local navigation. We re-identify the goal instance in egocentric vision using feature-matching and localize the goal instance by projecting matched features to a map. Each sub-task is solved using off-the-shelf components requiring zero fine-tuning. On the HM3D InstanceImageNav benchmark, this system outperforms a baseline end-to-end RL policy 7x and a state-of-the-art ImageNav model 2.3x (56% vs 25% success). We deploy this system to a mobile robot platform and demonstrate effective real-world performance, achieving an 88% success rate across a home and an office environment.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 3 Apr 2023.

Builds on

42 of 56 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2017Mask R-CNNcited by 7,259
  5. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  6. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  9. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  10. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  11. 2018On Evaluation of Embodied Navigation Agentscited by 526
  12. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
30 more
  1. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  2. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  3. 2018Deep Learning for Generic Object Detection: A Surveycited by 392
  4. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  5. 2020Learning to Explore using Active Neural SLAMcited by 308
  6. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  7. 2018Semi-parametric Topological Memory for Navigationcited by 203
  8. 2020Neural Topological SLAM for Visual Navigationcited by 193
  9. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
  10. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
  11. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  12. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  13. 2022Navigating to Objects in the Real Worldcited by 104
  14. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  15. 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 77
  16. 2022Offline Visual Representation Learning for Embodied Navigationcited by 74
  17. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 68
  18. 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
  19. 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 64
  20. 2021Instance-level Image Retrieval using Reranking Transformerscited by 63
  21. 2021No RL, No Simulation: Learning to Navigate without Navigatingcited by 63
  22. 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 55
  23. 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 53
  24. 2019Bayesian Relational Memory for Semantic Visual Navigationcited by 48
  25. 2022Retrospectives on the Embodied AI Workshopcited by 46
  26. 2021The Design of Stretch: A Compact, Lightweight Mobile Manipulator for Indoor Human Environmentscited by 39
  27. 2022Last-Mile Embodied Visual Navigationcited by 25
  28. 2022VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangementcited by 18
  29. 2022A Light Touch Approach to Teaching Transformers Multi-view Geometrycited by 6
  30. 2019re-OBJ: Jointly Learning the Foreground and Background for Object Instance Re-identificationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 18; 2025: 21; 2026: 6.

50 papers.

Sort:

By area:

  1. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 104
  2. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  3. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  4. 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 56
  5. 2024VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thoughtcited by 55
  6. 2024GaussNav: Gaussian Splatting for Visual Navigationcited by 49
  7. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 47
  8. 2024CityWalker: Learning Embodied Urban Navigation from Web-Scale Videoscited by 40
  9. 2025OctoNav: Towards Generalist Embodied Navigationcited by 33
  10. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
  11. 2024TANGO: Training-free Embodied AI Agents for Open-world Taskscited by 26
  12. 2023FGPrompt: Fine-grained Goal Prompting for Image-goal Navigationcited by 21
38 more
  1. 2023Selective Visual Representations Improve Convergence and Generalization for Embodied AIcited by 19
  2. 2023End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenoncited by 16
  3. 2024ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environmentscited by 16
  4. 2024Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigationcited by 15
  5. 2024Point Cloud Models Improve Visual Robustness in Robotic Learnerscited by 13
  6. 2025MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigationcited by 13
  7. 2025REGNav: Room Expert Guided Image-Goal Navigationcited by 12
  8. 2023What do we learn from a large-scale study of pre-trained visual representations in sim and real environments?cited by 11
  9. 2024Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environmentscited by 11
  10. 2024FloNa: Floor Plan Guided Embodied Visual Navigationcited by 10
  11. 2025IGL-Nav: Incremental 3D Gaussian Localization for Image-goal Navigationcited by 10
  12. 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10
  13. 2026VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memorycited by 10
  14. 2025LookOut: Real-World Humanoid Egocentric Navigationcited by 9
  15. 2024HELPER-X: A Unified Instructable Embodied Agent to Tackle Four Interactive Vision-Language Domains with Memory-Augmented Language Modelscited by 8
  16. 2025From reactive to cognitive: brain-inspired spatial intelligence for embodied agentscited by 7
  17. 2025PersONAL: Towards a Comprehensive Benchmark for Personalized Embodied Agentscited by 4
  18. 2024NOLO: Navigate Only Look Oncecited by 3
  19. 2025Image-Goal Navigation Using Refined Feature Guidance and Scene Graph Enhancementcited by 3
  20. 2025Scalable Multi-Task Reinforcement Learning for Generalizable Spatial Intelligence in Visuomotor Agentscited by 3
  21. 2025Wanderland: Geometrically Grounded Simulation for Open-World Embodied AIcited by 3
  22. 2025RANa: Retrieval-Augmented Navigationcited by 2
  23. 2025Hierarchical Scoring with 3D Gaussian Splatting for Instance Image-Goal Navigationcited by 2
  24. 2025What does really matter in image goal navigation?cited by 2
  25. 2025Kinaema: a recurrent sequence model for memory and pose in motioncited by 2
  26. 2025MG-Nav: Dual-Scale Visual Navigation via Sparse Spatial Memorycited by 2
  27. 2026LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigationcited by 2
  28. 2024Transformers for Image-Goal Navigationcited by 1
  29. 2024Diffusion as Reasoning: Enhancing Object Navigation via Diffusion Model Conditioned on LLM-based Object-Room Knowledgecited by 1
  30. 2025RSRNav: Reasoning Spatial Relationship for Image-Goal Navigationcited by 1
  31. 2025SplatSearch: Instance Image Goal Navigation for Mobile Robots using 3D Gaussian Splatting and Diffusion Modelscited by 1
  32. 2025ImagineNav++: Prompting Vision-Language Models as Embodied Navigator through Scene Imaginationcited by 1
  33. 2026ReasonNavi: Human-Inspired Global Map Reasoning for Zero-Shot Embodied Navigationcited by 1
  34. 2026EvoMemNav: Efficient Self-Evolving Fine-Grained Memory for Zero-Shot Embodied Navigationcited by 1
  35. 2024Real-world Instance-specific Image Goal Navigation: Bridging Domain Gaps via Contrastive Learningcited by 0
  36. 2025SkeNa: Learning to Navigate Unseen Environments Based on Abstract Hand-Drawn Mapscited by 0
  37. 2026Plug-and-Play Label Map Diffusion for Universal Goal-Oriented Navigationcited by 0
  38. 2026MuViSeg: Multi-View Segment Correspondences from Dense Geometry Priorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Object Goal Navigation with Recursive Implicit Maps
  2. 2026NaViRrator: Robot Navigation from Human-Readable Maps through a Learned Visual Route
  3. 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers
  4. 2026PlatonicNav: Unveiling Semantic Correspondence in Navigation with Platonic Topological Maps
  5. 2025FiLM-Nav: Efficient and Generalizable Navigation via VLM Fine-tuning
  6. 2025NaviTrace: Evaluating Embodied Navigation of Vision-Language Models
  7. 2025EmbodiedDiffusion: End-to-End Traversability-Guided Visual Diffusion for Heterogeneous Robot Navigation
  8. 2026IMPRINT: Image-Conditioned Query Enrichment for Long-Tail Object Goal Navigation