# VLM-GLoc: Vision-Language Model Enhanced Monte Carlo Localization for Robust Semantic Global Localization in Cluttered Quasi-Static Environments

arXiv: https://arxiv.org/abs/2605.30506 · HTML: https://mlsanity.com/paper/2605.30506/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 28 May 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 18 of 18)

- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2004.02857/) (2020)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning](https://mlsanity.com/paper/2309.16650/) (2023)
- [MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigation](https://mlsanity.com/paper/2401.07314/) (2024)
- [Precise Detection in Densely Packed Scenes](https://mlsanity.com/paper/1904.00853/) (2019)
- [Overconfidence is Key: Verbalized Uncertainty Evaluation in Large Language and Vision-Language Models](https://mlsanity.com/paper/2405.02917/) (2024)
- [A Survey on Global LiDAR Localization: Challenges, Advances and Open Problems](https://mlsanity.com/paper/2302.07433/) (2023)
- [Seeing is Believing, but How Much? A Comprehensive Analysis of Verbalized Calibration in Vision-Language Models](https://mlsanity.com/paper/2505.20236/) (2025)
- [IR-MCL: Implicit Representation-Based Online Global Localization](https://mlsanity.com/paper/2210.03113/) (2022)
- [Long-Term Localization using Semantic Cues in Floor Plan Maps](https://mlsanity.com/paper/2210.01456/) (2022)
- [Robust Onboard Localization in Changing Environments Exploiting Text Spotting](https://mlsanity.com/paper/2203.12647/) (2022)
- [Robust Lifelong Indoor LiDAR Localization using the Area Graph](https://mlsanity.com/paper/2308.05593/) (2023)
- [VLM-Loc: Localization in Point Cloud Maps via Vision-Language Models](https://mlsanity.com/paper/2603.09826/) (2026)
- [Probabilistic Object Maps for Long-Term Robot Localization](https://mlsanity.com/paper/2110.00128/) (2021)
- [Loc3R-VLM: Language-based Localization and 3D Reasoning with Vision-Language Models](https://mlsanity.com/paper/2603.18002/) (2026)
- [ShelfAware: Real-Time Semantic Localization in Quasi-Static Environments with Low-Cost Sensors](https://mlsanity.com/paper/2512.09065/) (2025)

## Abstract (arXiv)

Global localization in geometrically aliased, quasi-static environments such as grocery stores, offices, schools, and hospitals poses a significant challenge for mobile robots. Grocery stores with parallel aisles and a long tailed distribution of products, as well as offices and labs with repetitive furniture such as chairs, desks, monitors, and doors, exemplify common indoor environments that present geometric and even semantic ambiguity. Traditional approaches rely either on distinct geometric features or on domain-specific vision pipelines that struggle with long-tail semantic distributions and transient visual clutter. We present VLM-GLoc, a method for hierarchical semantic Monte Carlo Localization (MCL) that leverages open-vocabulary Vision-Language Models (VLMs) as a unified semantic observation front-end. We hypothesize a three-fold benefit from VLMs: (1) extracting highly discriminative rich text features, (2) implicit quality filtering of blurry or dynamic objects, and (3) permanence reasoning for targeted data augmentation. We introduce an inverse semantic proposal mechanism that seeds particles via text-to-map retrieval. Evaluated across two real-world environments with different characteristics and two different platforms: a 3,500 sq. ft. grocery store with a cellphone and a 3,700 sq. ft. lab space with a quadruped, VLM-GLoc achieves 70% and 74% global localization success respectively, substantially outperforming traditional geometry-only and domain-specific baselines.