# G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models

arXiv: https://arxiv.org/abs/2405.14702 · HTML: https://mlsanity.com/paper/2405.14702/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 49 references that are in the map.
- First posted on arXiv 23 May 2024.

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks](https://mlsanity.com/paper/2305.11175/) (2023)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval](https://mlsanity.com/paper/2011.12172/) (2020)
- [GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization](https://mlsanity.com/paper/2309.16020/) (2023)
- [Wide-Area Image Geolocalization with Aerial Reference Imagery](https://mlsanity.com/paper/1510.03743/) (2015)
- [Lending Orientation to Neural Networks for Cross-view Geo-localization](https://mlsanity.com/paper/1903.12351/) (2019)
- [TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization](https://mlsanity.com/paper/2204.00097/) (2022)
- [Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching](https://mlsanity.com/paper/2005.03860/) (2020)
- [PlaNet - Photo Geolocation with Convolutional Neural Networks](https://mlsanity.com/paper/1602.05314/) (2016)
- [R2Former: Unified Retrieval and Reranking Transformer for Place Recognition](https://mlsanity.com/paper/2304.03410/) (2023)
- [Revisiting IM2GPS in the Deep Learning Era](https://mlsanity.com/paper/1705.04838/) (2017)
- [Cross-View Image Matching for Geo-localization in Urban Environments](https://mlsanity.com/paper/1703.07815/) (2017)
- [CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps](https://mlsanity.com/paper/1808.02130/) (2018)
- [DiffTraj: Generating GPS Trajectory with Diffusion Probabilistic Model](https://mlsanity.com/paper/2304.11582/) (2023)
- [Instance-level Image Retrieval using Reranking Transformers](https://mlsanity.com/paper/2103.12236/) (2021)
- [Where in the World is this Image? Transformer-based Geo-localization in the Wild](https://mlsanity.com/paper/2204.13861/) (2022)
- [PIGEON: Predicting Image Geolocations](https://mlsanity.com/paper/2307.05845/) (2023)
- [Where We Are and What We're Looking At: Query Based Worldwide Image Geo-localization Using Hierarchies and Scenes](https://mlsanity.com/paper/2303.04249/) (2023)
- [ControlTraj: Controllable Trajectory Generation with Topology-Constrained Diffusion Model](https://mlsanity.com/paper/2404.15380/) (2024)
- [Correlation Verification for Image Retrieval](https://mlsanity.com/paper/2204.01458/) (2022)
- [Global Features are All You Need for Image Retrieval and Reranking](https://mlsanity.com/paper/2308.06954/) (2023)
- [Img2Loc: Revisiting Image Geolocalization using Multi-modality Foundation Models and Image-based Retrieval-Augmented Generation](https://mlsanity.com/paper/2403.19584/) (2024)
- [MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansion](https://mlsanity.com/paper/2310.19056/) (2023)
- [MLPST: MLP is All You Need for Spatio-Temporal Prediction](https://mlsanity.com/paper/2309.13363/) (2023)
- [PromptST: Prompt-Enhanced Spatio-Temporal Multi-Attribute Prediction](https://mlsanity.com/paper/2309.09500/) (2023)

## Built on by (most cited first, 30 of 34)

- [Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models](https://mlsanity.com/paper/2506.14674/) (2025)
- [Granular Privacy Control for Geolocation with Vision Language Models](https://mlsanity.com/paper/2407.04952/) (2024)
- [GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalization](https://mlsanity.com/paper/2505.13731/) (2025)
- [GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chains](https://mlsanity.com/paper/2505.18700/) (2025)
- [Swarm Intelligence in Geo-Localization: A Multi-Agent Large Vision-Language Model Collaborative Framework](https://mlsanity.com/paper/2408.11312/) (2024)
- [Towards Interactive Global Geolocation Assistant](https://mlsanity.com/paper/2412.08907/) (2024)
- [Scaling Image Geo-Localization to Continent Level](https://mlsanity.com/paper/2510.26795/) (2025)
- [GeoLocSFT: Efficient Visual Geolocation via Supervised Fine-Tuning of Multimodal Foundation Models](https://mlsanity.com/paper/2506.01277/) (2025)
- [Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalization](https://mlsanity.com/paper/2601.05432/) (2026)
- [NAVIG: Natural Language-guided Analysis with Vision Language Models for Image Geo-localization](https://mlsanity.com/paper/2502.14638/) (2025)
- [GeoToken: Hierarchical Geolocalization of Images via Next Token Prediction](https://mlsanity.com/paper/2511.01082/) (2025)
- [GeoAgent: Learning to Geolocate Everywhere with Reinforced Geographic Characteristics](https://mlsanity.com/paper/2602.12617/) (2026)
- [TAPO: Task-Referenced Adaptation for Prompt Optimization](https://mlsanity.com/paper/2501.06689/) (2025)
- [Towards Interpretable Geo-localization: a Concept-Aware Global Image-GPS Alignment Framework](https://mlsanity.com/paper/2509.01910/) (2025)
- [VLM-Guided Visual Place Recognition for Planet-Scale Geo-Localization](https://mlsanity.com/paper/2507.17455/) (2025)
- [GeoSURGE: Geo-localization using Semantic Fusion with Hierarchy of Geographic Embeddings](https://mlsanity.com/paper/2510.01448/) (2025)
- [Emotion and Intention Guided Multi-Modal Learning for Sticker Response Selection](https://mlsanity.com/paper/2511.17587/) (2025)
- [REVERSE: Reinforcing Evidence Verification and Search for Agentic Image geo-localization](https://mlsanity.com/paper/2605.26861/) (2026)
- [GEO-Detective: Unveiling Location Privacy Risks in Images with LLM Agents](https://mlsanity.com/paper/2511.22441/) (2025)
- [LocationAgent: A Hierarchical Agent for Image Geolocation via Decoupling Strategy and Evidence from Parametric Knowledge](https://mlsanity.com/paper/2601.19155/) (2026)
- [VidTAG: Temporally Aligned Video to GPS Geolocalization with Denoising Sequence Prediction at a Global Scale](https://mlsanity.com/paper/2604.12159/) (2026)
- [DualGeo: A Dual-View Framework for Worldwide Image Geo-localization](https://mlsanity.com/paper/2604.25533/) (2026)
- [VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing](https://mlsanity.com/paper/2512.11490/) (2025)
- [GeoRouter: Dynamic Paradigm Routing for Worldwide Image Geolocalization](https://mlsanity.com/paper/2603.24376/) (2026)
- [GeoSearch: Augmenting Worldwide Geolocalization with Web-Scale Reverse Image Search and Image Matching](https://mlsanity.com/paper/2604.25390/) (2026)
- [Pinpoint: Grounded Worldwide Image Geolocation via Cross-Source Retrieval and Reranking](https://mlsanity.com/paper/2606.04133/) (2026)
- [GeoGNN: Time Series Geo-Localization using Two-Tower Graph Neural Networks](https://mlsanity.com/paper/2606.08303/) (2026)
- [GeoMetric: Injecting Metric Geographic Structure into Worldwide Image Geo-Localization](https://mlsanity.com/paper/2606.08918/) (2026)
- [Defending from GeoLocalization through Adversarial Road Trips](https://mlsanity.com/paper/2607.03277/) (2026)
- [GeoSEAN: Explainable Country-Level Image Geolocation for ASEAN Regions](https://mlsanity.com/paper/2607.12284/) (2026)

## Abstract (arXiv)

Worldwide geolocalization aims to locate the precise location at the coordinate level of photos taken anywhere on the Earth. It is very challenging due to 1) the difficulty of capturing subtle location-aware visual semantics, and 2) the heterogeneous geographical distribution of image data. As a result, existing studies have clear limitations when scaled to a worldwide context. They may easily confuse distant images with similar visual contents, or cannot adapt to various locations worldwide with different amounts of relevant data. To resolve these limitations, we propose G3, a novel framework based on Retrieval-Augmented Generation (RAG). In particular, G3 consists of three steps, i.e., Geo-alignment, Geo-diversification, and Geo-verification to optimize both retrieval and generation phases of worldwide geolocalization. During Geo-alignment, our solution jointly learns expressive multi-modal representations for images, GPS and textual descriptions, which allows us to capture location-aware semantics for retrieving nearby images for a given query. During Geo-diversification, we leverage a prompt ensembling method that is robust to inconsistent retrieval performance for different image queries. Finally, we combine both retrieved and generated GPS candidates in Geo-verification for location prediction. Experiments on two well-established datasets IM2GPS3k and YFCC4k verify the superiority of G3 compared to other state-of-the-art methods. Our code and data are available online for reproduction.