# FM-Loc: Using Foundation Models for Improved Vision-based Localization

arXiv: https://arxiv.org/abs/2304.07058 · HTML: https://mlsanity.com/paper/2304.07058/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2023.

## Built on by (most cited first, 11 of 11)

- [AnyLoc: Towards Universal Visual Place Recognition](https://mlsanity.com/paper/2308.00688/) (2023)
- [Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis](https://mlsanity.com/paper/2312.08782/) (2023)
- [What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Models](https://mlsanity.com/paper/2512.03422/) (2025)
- [Revisit Anything: Visual Place Recognition via Image Segment Retrieval](https://mlsanity.com/paper/2409.18049/) (2024)
- [Multimodal LLM Guided Exploration and Active Mapping using Fisher Information](https://mlsanity.com/paper/2410.17422/) (2024)
- [CLIP-Loc: Multi-modal Landmark Association for Global Localization in Object-based Maps](https://mlsanity.com/paper/2402.06092/) (2024)
- [Tell Me Where You Are: Multimodal LLMs Meet Place Recognition](https://mlsanity.com/paper/2406.17520/) (2024)
- [CLIP-Clique: Graph-based Correspondence Matching Augmented by Vision Language Models for Object-based Global Localization](https://mlsanity.com/paper/2410.03054/) (2024)
- [Context-Based Visual-Language Place Recognition](https://mlsanity.com/paper/2410.19341/) (2024)
- [Polar Perspectives: Evaluating 2-D LiDAR Projections for Robust Place Recognition with Visual Foundation Models](https://mlsanity.com/paper/2512.02897/) (2025)
- [MVL-Loc: Leveraging Vision-Language Model for Generalizable Multi-Scene Camera Relocalization](https://mlsanity.com/paper/2507.04509/) (2025)

## Abstract (arXiv)

Visual place recognition is essential for vision-based robot localization and SLAM. Despite the tremendous progress made in recent years, place recognition in changing environments remains challenging. A promising approach to cope with appearance variations is to leverage high-level semantic features like objects or place categories. In this paper, we propose FM-Loc which is a novel image-based localization approach based on Foundation Models that uses the Large Language Model GPT-3 in combination with the Visual-Language Model CLIP to construct a semantic image descriptor that is robust to severe changes in scene geometry and camera viewpoint. We deploy CLIP to detect objects in an image, GPT-3 to suggest potential room labels based on the detected objects, and CLIP again to propose the most likely location label. The object labels and the scene label constitute an image descriptor that we use to calculate a similarity score between the query and database images. We validate our approach on real-world data that exhibit significant changes in camera viewpoints and object placement between the database and query trajectories. The experimental results demonstrate that our method is applicable to a wide range of indoor scenarios without the need for training or fine-tuning.