IMPRINT: Image-Conditioned Query Enrichment for Long-Tail Object Goal Navigation
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Embodied AI increasingly relies on queryable semantic maps built from pre-trained vision-language models to enable zero-shot Object Goal Navigation (ObjectNav). However, existing approaches typically depend on text-only queries, which become less reliable as semantic specificity increases toward fine-grained object categories. We introduce IMPRINT, a zero-shot plug-and-play framework that enriches textual object queries with web-sourced images to improve grounding in queryable maps. Retrieved images are encoded using a vision-language model, matched against the semantic map to produce similarity maps, and aggregated to yield context-aware localization. Notably, this requires no training or modification of the underlying navigation policy. To explicitly evaluate long-tail behavior, we present HSSD-rare, a new ObjectNav benchmark built on Habitat Synthetic Scenes and featuring semantically specific subcategories. Across both OVON and HSSD-rare, image-conditioned queries consistently improve object grounding and yield end-to-end navigation gains. Further analysis reveals that translating localization gains to navigation performance depends critically on downstream detection quality, highlighting a key systems bottleneck in long-tail embodied navigation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 21 references that are in the map. First posted on arXiv 27 Jul 2026.
Builds on
19 of 21 references.
Closest ancestors · 1
Same direction, newest first
Other references · 18
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
- 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
- 2021Deep Long-Tailed Learning: A Surveycited by 417
- 2023Scaling Open-Vocabulary Object Detectioncited by 343
- 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
6 more
- 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
- 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 97
- 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
- 2023Open-Fusion: Real-time Open-Vocabulary 3D Mapping and Queryable Scene Representationcited by 54
- 2025VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Searchcited by 35
- 2024Search and Detect: Training-Free Long Tail Object Detection via Web-Image Retrievalcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigation
- 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers
- 2024Prioritized Semantic Learning for Zero-shot Instance Navigation
- 2024Open-Vocabulary Object-Goal Navigation by Generalizing Semantic Mapping with Dense CLIP
- 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models
- 2026Finder: Agentic Closed-Loop Object Finding for Embodied Grounding
- 2025osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning
- 2025SemNav: A Model-Based Planner for Zero-Shot Object Goal Navigation Using Vision-Foundation Models