mlsanityData to 14 Oct 2026

IMPRINT: Image-Conditioned Query Enrichment for Long-Tail Object Goal Navigation

arXiv 2607.25106 ↗First posted 27 Jul 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Embodied AI increasingly relies on queryable semantic maps built from pre-trained vision-language models to enable zero-shot Object Goal Navigation (ObjectNav). However, existing approaches typically depend on text-only queries, which become less reliable as semantic specificity increases toward fine-grained object categories. We introduce IMPRINT, a zero-shot plug-and-play framework that enriches textual object queries with web-sourced images to improve grounding in queryable maps. Retrieved images are encoded using a vision-language model, matched against the semantic map to produce similarity maps, and aggregated to yield context-aware localization. Notably, this requires no training or modification of the underlying navigation policy. To explicitly evaluate long-tail behavior, we present HSSD-rare, a new ObjectNav benchmark built on Habitat Synthetic Scenes and featuring semantically specific subcategories. Across both OVON and HSSD-rare, image-conditioned queries consistently improve object grounding and yield end-to-end navigation gains. Further analysis reveals that translating localization gains to navigation performance depends critically on downstream detection quality, highlighting a key systems bottleneck in long-tail embodied navigation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 21 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

19 of 21 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation6 Dec

Other references · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  6. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  7. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  8. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
  9. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  10. 2021Deep Long-Tailed Learning: A Surveycited by 417
  11. 2023Scaling Open-Vocabulary Object Detectioncited by 343
  12. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 178
6 more
  1. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 167
  2. 2023SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentationcited by 97
  3. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  4. 2023Open-Fusion: Real-time Open-Vocabulary 3D Mapping and Queryable Scene Representationcited by 54
  5. 2025VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Searchcited by 35
  6. 2024Search and Detect: Training-Free Long Tail Object Detection via Web-Image Retrievalcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigation
  2. 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontiers
  3. 2024Prioritized Semantic Learning for Zero-shot Instance Navigation
  4. 2024Open-Vocabulary Object-Goal Navigation by Generalizing Semantic Mapping with Dense CLIP
  5. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models
  6. 2026Finder: Agentic Closed-Loop Object Finding for Embodied Grounding
  7. 2025osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning
  8. 2025SemNav: A Model-Based Planner for Zero-Shot Object Goal Navigation Using Vision-Foundation Models