# Efficient Navigation in Unknown Indoor Environments with Vision-Language Models

arXiv: https://arxiv.org/abs/2510.04991 · HTML: https://mlsanity.com/paper/2510.04991/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 36 of its 39 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 36)

- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models](https://mlsanity.com/paper/2212.04088/) (2022)
- [LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action](https://mlsanity.com/paper/2207.04429/) (2022)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [A Survey on Multimodal Large Language Models for Autonomous Driving](https://mlsanity.com/paper/2311.12320/) (2023)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation](https://mlsanity.com/paper/2402.15852/) (2024)
- [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023)
- [VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation](https://mlsanity.com/paper/2312.03275/) (2023)
- [Physically Grounded Vision-Language Models for Robotic Manipulation](https://mlsanity.com/paper/2309.02561/) (2023)
- [Video Language Planning](https://mlsanity.com/paper/2310.10625/) (2023)
- [Open-vocabulary Queryable Scene Representations for Real World Planning](https://mlsanity.com/paper/2209.09874/) (2022)
- [Occupancy Anticipation for Efficient Exploration and Navigation](https://mlsanity.com/paper/2008.09285/) (2020)
- [Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planning](https://mlsanity.com/paper/2310.10103/) (2023)
- [ViNG: Learning Open-World Navigation with Visual Goals](https://mlsanity.com/paper/2012.09812/) (2020)
- [Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs](https://mlsanity.com/paper/2407.07775/) (2024)
- [Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents](https://mlsanity.com/paper/2303.00855/) (2023)
- [ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints](https://mlsanity.com/paper/2202.11271/) (2022)
- [PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators](https://mlsanity.com/paper/2406.20083/) (2024)
- [EGO-Planner: An ESDF-free Gradient-based Local Planner for Quadrotors](https://mlsanity.com/paper/2008.08835/) (2020)
- [Context-Aware Planning and Environment-Aware Memory for Instruction Following Embodied Agents](https://mlsanity.com/paper/2308.07241/) (2023)
- [LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videos](https://mlsanity.com/paper/2410.03603/) (2024)
- [REAL: Resilience and Adaptation using Large Language Models on Autonomous Aerial Robots](https://mlsanity.com/paper/2311.01403/) (2023)
- [FASTER: Fast and Safe Trajectory Planner for Navigation in Unknown Environments](https://mlsanity.com/paper/2001.04420/) (2020)
- [Tag Map: A Text-Based Map for Spatial Reasoning and Navigation with Large Language Models](https://mlsanity.com/paper/2409.15451/) (2024)
- [MapEx: Indoor Structure Exploration with Probabilistic Information Gain from Global Map Predictions](https://mlsanity.com/paper/2409.15590/) (2024)
- [Multimodal LLM Guided Exploration and Active Mapping using Fisher Information](https://mlsanity.com/paper/2410.17422/) (2024)
- [3P-LLM: Probabilistic Path Planning using Large Language Model for Autonomous Robot Navigation](https://mlsanity.com/paper/2403.18778/) (2024)

## Built on by (most cited first, 2 of 2)

- [Contextual Safety Reasoning and Grounding for Open-World Robots](https://mlsanity.com/paper/2602.19983/) (2026)
- [CoFineLLM: Conformal Finetuning of LLMs for Language-Instructed Robot Planning](https://mlsanity.com/paper/2511.06575/) (2025)

## Abstract (arXiv)

We present a novel high-level planning framework that leverages vision-language models (VLMs) to improve autonomous navigation in unknown indoor environments with many dead ends. Traditional exploration methods often take inefficient routes due to limited global reasoning and reliance on local heuristics. In contrast, our approach enables a VLM to reason directly about occupancy maps in a zero-shot manner, selecting subgoals that are likely to yield more efficient paths. At each planning step, we convert a 3D occupancy grid into a partial 2D map of the environment, and generate candidate subgoals. Each subgoal is then evaluated and ranked against other candidates by the model. We integrate this planning scheme into DYNUS citekondo2025dynus, a state-of-the-art trajectory planner, and demonstrate improved navigation efficiency in simulation. The VLM infers structural patterns (e.g., rooms, corridors) from incomplete maps and balances the need to make progress toward a goal against the risk of entering unknown space. This reduces common greedy failures (e.g., detouring into small rooms) and achieves about 10% shorter paths on average.