Turning Generators into Retrievers: Unlocking MLLMs for Natural Language-Guided Geo-Localization
Not on the map. Why?
Abstract (arXiv)
Natural-language Guided Cross-view Geo-localization (NGCG) aims to retrieve geo-tagged satellite imagery using textual descriptions of ground scenes. While recent NGCG methods commonly rely on CLIP-style dual-encoder architectures, they often suffer from weak cross-modal generalization and require complex architectural designs. In contrast, Multimodal Large Language Models (MLLMs) offer powerful semantic reasoning capabilities but are not directly optimized for retrieval tasks. In this work, we present a simple yet effective framework to adapt MLLMs for NGCG via parameter-efficient finetuning. Our approach optimizes latent representations within the MLLM while preserving its pretrained multimodal knowledge, enabling strong cross-modal alignment without redesigning model architectures. Through systematic analysis of diverse variables, from model backbone to feature aggregation, we provide practical and generalizable insights for leveraging MLLMs in NGCG. Our method achieves SOTA on GeoText-1652 with a 12.2% improvement in Text-to-Image Recall@1 and secures top performance in 5 out of 12 subtasks on CVG-Text, all while surpassing baselines with far fewer trainable parameters. These results position MLLMs as a robust foundation for semantic cross-view retrieval and pave the way for MLLM-based NGCG to be adopted as a scalable, powerful alternative to traditional dual-encoder designs. Project page and code are available at https://yuqichen888.github.io/NGCG-MLLMs-web/.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 47 of its 62 references that are in the map. First posted on arXiv 12 Apr 2026.
Builds on
47 of 62 references.
References · 47
- 2017Attention Is All You Needcited by 59,932
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2023Visual Instruction Tuningcited by 7,792
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2020End-to-End Object Detection with Transformerscited by 7,384
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
35 more
- 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2024Phi-4 Technical Reportcited by 675
- 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
- 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
- 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
- 2021Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Conceptscited by 255
- 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
- 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
- 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 202
- 2020VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrievalcited by 188
- 2015Wide-Area Image Geolocalization with Aerial Reference Imagerycited by 185
- 2019Lending Orientation to Neural Networks for Cross-view Geo-localizationcited by 179
- 2022TransGeo: Transformer Is All You Need for Cross-view Image Geo-localizationcited by 166
- 2020University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localizationcited by 152
- 2019Optimal Feature Transport for Cross-View Image Geo-Localizationcited by 90
- 2018Cross-View Image Synthesis using Conditional GANscited by 69
- 2022Cross-view Geo-localization via Learning Disentangled Geometric Layout Correspondencecited by 69
- 2023Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Studycited by 64
- 2024Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Drivingcited by 53
- 2023Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matchingcited by 40
- 2024Cross-view image geo-localization with Panorama-BEV Co-Retrieval Networkcited by 40
- 2024Where am I? Cross-View Geo-localization with Natural Language Descriptionscited by 38
- 2023Simple, Effective and General: A New Backbone for Cross-view Image Geo-localizationcited by 36
- 2022Cross-View Image Sequence Geo-localizationcited by 31
- 2023GeoDTR+: Toward generic cross-view geolocalization via geometric disentanglementcited by 27
- 2022Cross-View Panorama Image Synthesiscited by 26
- 2024Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectivescited by 22
- 2024Multi-modal Vision Pre-training for Medical Image Analysiscited by 22
- 2024Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encodercited by 11
- 2024Cross-View Meets Diffusion: Aerial Image Synthesis with Geometry and Text Guidancecited by 9
- 2025VICI: VLM-Instructed Cross-view Image-localisationcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026What Words Keep of a Place: Zero-Shot Language Reasoning for Cross-View Geo-Localization
- 2025Foundation Models for Remote Sensing: An Analysis of MLLMs for Object Localization
- 2026From Coordinates to Candidate Regions: Temporal Change Localization via Region Selection in Remote Sensing Multimodal LLMs
- 2026GTPred: Benchmarking MLLMs for Interpretable Geo-localization and Time-of-capture Prediction
- 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
- 2025GeoZero: Incentivizing Reasoning from Scratch on Geospatial Scenes
- 2025A Parameter-Efficient Mixture-of-Experts Framework for Cross-Modal Geo-Localization
- 2026Where Do Vision-Language Models Fail? World Scale Analysis for Image Geolocalization