mlsanityData to 7 Oct 2026

Turning Generators into Retrievers: Unlocking MLLMs for Natural Language-Guided Geo-Localization

arXiv 2604.10721 ↗First posted 12 Apr 2026cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Natural-language Guided Cross-view Geo-localization (NGCG) aims to retrieve geo-tagged satellite imagery using textual descriptions of ground scenes. While recent NGCG methods commonly rely on CLIP-style dual-encoder architectures, they often suffer from weak cross-modal generalization and require complex architectural designs. In contrast, Multimodal Large Language Models (MLLMs) offer powerful semantic reasoning capabilities but are not directly optimized for retrieval tasks. In this work, we present a simple yet effective framework to adapt MLLMs for NGCG via parameter-efficient finetuning. Our approach optimizes latent representations within the MLLM while preserving its pretrained multimodal knowledge, enabling strong cross-modal alignment without redesigning model architectures. Through systematic analysis of diverse variables, from model backbone to feature aggregation, we provide practical and generalizable insights for leveraging MLLMs in NGCG. Our method achieves SOTA on GeoText-1652 with a 12.2% improvement in Text-to-Image Recall@1 and secures top performance in 5 out of 12 subtasks on CVG-Text, all while surpassing baselines with far fewer trainable parameters. These results position MLLMs as a robust foundation for semantic cross-view retrieval and pave the way for MLLM-based NGCG to be adopted as a scalable, powerful alternative to traditional dual-encoder designs. Project page and code are available at https://yuqichen888.github.io/NGCG-MLLMs-web/.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 47 of its 62 references that are in the map. First posted on arXiv 12 Apr 2026.

Builds on

47 of 62 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  11. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  12. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
35 more
  1. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  2. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  4. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  6. 2024Phi-4 Technical Reportcited by 675
  7. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  8. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  9. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  10. 2021Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Conceptscited by 255
  11. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 226
  12. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  13. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 202
  14. 2020VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrievalcited by 188
  15. 2015Wide-Area Image Geolocalization with Aerial Reference Imagerycited by 185
  16. 2019Lending Orientation to Neural Networks for Cross-view Geo-localizationcited by 179
  17. 2022TransGeo: Transformer Is All You Need for Cross-view Image Geo-localizationcited by 166
  18. 2020University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localizationcited by 152
  19. 2019Optimal Feature Transport for Cross-View Image Geo-Localizationcited by 90
  20. 2018Cross-View Image Synthesis using Conditional GANscited by 69
  21. 2022Cross-view Geo-localization via Learning Disentangled Geometric Layout Correspondencecited by 69
  22. 2023Adaptive Latent Diffusion Model for 3D Medical Image to Image Translation: Multi-modal Magnetic Resonance Imaging Studycited by 64
  23. 2024Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Drivingcited by 53
  24. 2023Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matchingcited by 40
  25. 2024Cross-view image geo-localization with Panorama-BEV Co-Retrieval Networkcited by 40
  26. 2024Where am I? Cross-View Geo-localization with Natural Language Descriptionscited by 38
  27. 2023Simple, Effective and General: A New Backbone for Cross-view Image Geo-localizationcited by 36
  28. 2022Cross-View Image Sequence Geo-localizationcited by 31
  29. 2023GeoDTR+: Toward generic cross-view geolocalization via geometric disentanglementcited by 27
  30. 2022Cross-View Panorama Image Synthesiscited by 26
  31. 2024Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectivescited by 22
  32. 2024Multi-modal Vision Pre-training for Medical Image Analysiscited by 22
  33. 2024Exploring How Generative MLLMs Perceive More Than CLIP with the Same Vision Encodercited by 11
  34. 2024Cross-View Meets Diffusion: Aerial Image Synthesis with Geometry and Text Guidancecited by 9
  35. 2025VICI: VLM-Instructed Cross-view Image-localisationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026What Words Keep of a Place: Zero-Shot Language Reasoning for Cross-View Geo-Localization
  2. 2025Foundation Models for Remote Sensing: An Analysis of MLLMs for Object Localization
  3. 2026From Coordinates to Candidate Regions: Temporal Change Localization via Region Selection in Remote Sensing Multimodal LLMs
  4. 2026GTPred: Benchmarking MLLMs for Interpretable Geo-localization and Time-of-capture Prediction
  5. 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?
  6. 2025GeoZero: Incentivizing Reasoning from Scratch on Geospatial Scenes
  7. 2025A Parameter-Efficient Mixture-of-Experts Framework for Cross-Modal Geo-Localization
  8. 2026Where Do Vision-Language Models Fail? World Scale Analysis for Image Geolocalization