mlsanityData to 13 Oct 2026

HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigation

arXiv 2409.14296 ↗First posted 22 Sep 2024cs.AI cs.ROcited by 71

Not on the map. Why?

Abstract (arXiv)

We present the Habitat-Matterport 3D Open Vocabulary Object Goal Navigation dataset (HM3D-OVON), a large-scale benchmark that broadens the scope and semantic range of prior Object Goal Navigation (ObjectNav) benchmarks. Leveraging the HM3DSem dataset, HM3D-OVON incorporates over 15k annotated instances of household objects across 379 distinct categories, derived from photo-realistic 3D scans of real-world environments. In contrast to earlier ObjectNav datasets, which limit goal objects to a predefined set of 6-20 categories, HM3D-OVON facilitates the training and evaluation of models with an open-set of goals defined through free-form language at test-time. Through this open-vocabulary formulation, HM3D-OVON encourages progress towards learning visuo-semantic navigation behaviors that are capable of searching for any object specified by text in an open-vocabulary manner. Additionally, we systematically evaluate and compare several different types of approaches on HM3D-OVON. We find that HM3D-OVON can be used to train an open-vocabulary ObjectNav agent that achieves both higher performance and is more robust to localization and actuation noise than the state-of-the-art ObjectNav approach. We hope that our benchmark and baseline results will drive interest in developing embodied agents that can navigate real-world spaces to find household objects specified through free-form language, taking a step towards more flexible and human-like semantic visual navigation. Code and videos available at: naoki.io/ovon.

Key facts. Cited by 71 papers in the map (data through 13 Oct 2026). Builds on 31 of its 42 references that are in the map. First posted on arXiv 22 Sep 2024.

Builds on

31 of 42 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,106
  2. 2017Proximal Policy Optimization Algorithmscited by 14,592
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,299
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,068
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,360
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,169
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,016
  9. 2018On Evaluation of Embodied Navigation Agentscited by 531
  10. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  11. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 371
  12. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 353
19 more
  1. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  2. 2023Scaling Open-Vocabulary Object Detectioncited by 337
  3. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  4. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  5. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
  6. 2020Neural Topological SLAM for Visual Navigationcited by 193
  7. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  8. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  9. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 164
  10. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
  11. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  12. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  13. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  14. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 92
  15. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 68
  16. 2023Navigating to Objects Specified by Imagescited by 50
  17. 2019PyRobot: An Open-source Robotics Framework for Research and Benchmarkingcited by 39
  18. 2021The Design of Stretch: A Compact, Lightweight Mobile Manipulator for Indoor Human Environmentscited by 39
  19. 2022VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangementcited by 18

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 7; 2025: 26; 2026: 38.

71 papers.

Sort:

By area:

  1. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 149
  2. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 84
  3. 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigationcited by 50
  4. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 43
  5. 2025OctoNav: Towards Generalist Embodied Navigationcited by 33
  6. 2024TANGO: Training-free Embodied AI Agents for Open-world Taskscited by 26
  7. 2026ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigationcited by 24
  8. 2025AstraNav-World: World Model for Foresight Control and Consistencycited by 17
  9. 2025Towards Autonomous Micromobility through Scalable Urban Simulationcited by 16
  10. 2025A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AIcited by 16
  11. 2025Towards Physically Executable 3D Gaussian for Embodied Navigationcited by 16
  12. 2024EfficientEQA: An Efficient Approach to Open-Vocabulary Embodied Question Answeringcited by 15
38 more
  1. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 15
  2. 2025MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigationcited by 13
  3. 2026Explore with Long-term Memory: A Benchmark and Multimodal LLM-based Reinforcement Learning Framework for Embodied Explorationcited by 13
  4. 2026Qwen-RobotNav Technical Report: A Scalable Navigation Model Designed for an Agentic Navigation Systemcited by 11
  5. 2026VLingNav: Embodied Navigation with Adaptive Reasoning and Visual-Assisted Linguistic Memorycited by 10
  6. 2025WoMAP: World Models For Embodied Open-Vocabulary Object Localizationcited by 9
  7. 2025LookOut: Real-World Humanoid Egocentric Navigationcited by 9
  8. 2025Nav-R2 Dual-Relation Reasoning for Generalizable Open-Vocabulary Object-Goal Navigationcited by 9
  9. 2024Collaborative Instance Object Navigation: Leveraging Uncertainty-Awareness to Minimize Human-Agent Dialoguescited by 7
  10. 2025From reactive to cognitive: brain-inspired spatial intelligence for embodied agentscited by 7
  11. 2025UrbanNav: Learning Language-Guided Urban Navigation from Web-Scale Human Trajectoriescited by 6
  12. 2024DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenescited by 5
  13. 2024ObjectFinder: An Open-Vocabulary Assistive System for Interactive Object Search by Blind Peoplecited by 5
  14. 2025EfficientNav: Towards On-Device Object-Goal Navigation with Navigation Map Caching and Retrievalcited by 5
  15. 2025Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Explorationcited by 5
  16. 2025D3D-VLP: Dynamic 3D Vision-Language-Planning Model for Embodied Grounding and Navigationcited by 5
  17. 2026LongNav-R1: Horizon-Adaptive Multi-Turn RL for Long-Horizon VLA Navigationcited by 5
  18. 2026CoMaTrack: Competitive Multi-Agent Game-Theoretic Tracking with Vision-Language-Action Modelscited by 5
  19. 2026ABot-N1: Toward a General Visual Language Navigation Foundation Modelcited by 5
  20. 2025PersONAL: Towards a Comprehensive Benchmark for Personalized Embodied Agentscited by 4
  21. 2025Deploying Foundation Model-Enabled Air and Ground Robots in the Field: Challenges and Opportunitiescited by 3
  22. 2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
  23. 2026Bridging the Indoor-Outdoor Gap: Vision-Centric Instruction-Guided Embodied Navigation for the Last Meterscited by 3
  24. 2026OpenFrontier: General Navigation with Visual-Language Grounded Frontierscited by 3
  25. 2026Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigationcited by 3
  26. 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoningcited by 3
  27. 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harnesscited by 3
  28. 2025HyPerNav: Hybrid Perception for Object-Oriented Navigation in Unknown Environmentcited by 2
  29. 2026LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigationcited by 2
  30. 2026MerNav: A Highly Generalizable Memory-Execute-Review Framework for Zero-Shot Object Goal Navigationcited by 2
  31. 2026SpaAct: Spatially-Activated Transition Learning with Curriculum Adaptation for Vision-Language Navigationcited by 2
  32. 2026POINav: Benchmarking and Enhancing Final-Meters Arrival in Real-World Vision-Language Navigationcited by 2
  33. 2026Planning with the Viewscited by 2
  34. 2026LIME: Learning Intent-aware Camera Motion from Egocentric Videocited by 2
  35. 2025MLFM: Multi-Layered Feature Maps for Richer Language Understanding in Zero-Shot Semantic Navigationcited by 1
  36. 2025FOM-Nav: Frontier-Object Maps for Object Goal Navigationcited by 1
  37. 2026EmbodiedSplat: Online Feed-Forward Semantic 3DGS for Open-Vocabulary 3D Scene Understandingcited by 1
  38. 2026GoalVLM: VLM-driven Object Goal Navigation for Multi-Agent Systemcited by 1

50 most cited of 71 All 71 (JSON)