# OpenMap: Instruction Grounding via Open-Vocabulary Visual-Language Mapping

arXiv: https://arxiv.org/abs/2508.01723 · HTML: https://mlsanity.com/paper/2508.01723/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Aug 2025.

## Built on by (most cited first, 2 of 2)

- [MapDream: Task-Driven Map Learning for Vision-Language Navigation](https://mlsanity.com/paper/2602.00222/) (2026)
- [Instance-Enriched Semantic Maps for Visual Language Navigation](https://mlsanity.com/paper/2607.12630/) (2026)

## Abstract (arXiv)

Grounding natural language instructions to visual observations is fundamental for embodied agents operating in open-world environments. Recent advances in visual-language mapping have enabled generalizable semantic representations by leveraging vision-language models (VLMs). However, these methods often fall short in aligning free-form language commands with specific scene instances, due to limitations in both instance-level semantic consistency and instruction interpretation. We present OpenMap, a zero-shot open-vocabulary visual-language map designed for accurate instruction grounding in navigation tasks. To address semantic inconsistencies across views, we introduce a Structural-Semantic Consensus constraint that jointly considers global geometric structure and vision-language similarity to guide robust 3D instance-level aggregation. To improve instruction interpretation, we propose an LLM-assisted Instruction-to-Instance Grounding module that enables fine-grained instance selection by incorporating spatial context and expressive target descriptions. We evaluate OpenMap on ScanNet200 and Matterport3D, covering both semantic mapping and instruction-to-target retrieval tasks. Experimental results show that OpenMap outperforms state-of-the-art baselines in zero-shot settings, demonstrating the effectiveness of our method in bridging free-form language and 3D perception for embodied navigation.