mlsanityData to 13 Oct 2026

Fine-Tuning Vision-Language Models for Visual Navigation Assistance

arXiv 2509.07488 ↗First posted 9 Sep 2025cs.CV cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

We address vision-language-driven indoor navigation to assist visually impaired individuals in reaching a target location using images and natural language guidance. Traditional navigation systems are ineffective indoors due to the lack of precise location data. Our approach integrates vision and language models to generate step-by-step navigational instructions, enhancing accessibility and independence. We fine-tune the BLIP-2 model with Low Rank Adaptation (LoRA) on a manually annotated indoor navigation dataset. We propose an evaluation metric that refines the BERT F1 score by emphasizing directional and sequential variables, providing a more comprehensive measure of navigational performance. After applying LoRA, the model significantly improved in generating directional instructions, overcoming limitations in the original BLIP-2 model.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 8 of its 16 references that are in the map. First posted on arXiv 9 Sep 2025.

Builds on

8 of 16 references.

References · 8

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
  2. 2023Visual Instruction Tuningcited by 7,827
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,363
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,898
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,855
  6. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,267
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,016
  8. 2024Navigating the Nuances: A Fine-grained Evaluation of Vision-Language Navigationcited by 7

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigationcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2025A Navigation Framework Utilizing Vision-Language Models
  2. 2023Prompt-based Context- and Domain-aware Pretraining for Vision and Language Navigation
  3. 2023Accessible Instruction-Following Agent
  4. 2026Exploring the Use of VLMs for Navigation Assistance for People with Blindness and Low Vision
  5. 2026Small yet Assistive: Spatially-Aware Post-Training for Low Vision
  6. 2026LangMap: A Human-Verified Benchmark for Hierarchical Open-Vocabulary Goal Navigation
  7. 2025Floorplan2Guide: LLM-Guided Floorplan Parsing for BLV Indoor Navigation
  8. 2024TINA: Think, Interaction, and Action Framework for Zero-Shot Vision Language Navigation