# Where We Are and What We're Looking At: Query Based Worldwide Image Geo-localization Using Hierarchies and Scenes

arXiv: https://arxiv.org/abs/2303.04249 · HTML: https://mlsanity.com/paper/2303.04249/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval](https://mlsanity.com/paper/2011.12172/) (2020)
- [TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization](https://mlsanity.com/paper/2204.00097/) (2022)
- [Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching](https://mlsanity.com/paper/2005.03860/) (2020)
- [PlaNet - Photo Geolocation with Convolutional Neural Networks](https://mlsanity.com/paper/1602.05314/) (2016)
- [Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localization](https://mlsanity.com/paper/2103.06818/) (2021)
- [Bridging the Domain Gap for Ground-to-Aerial Image Matching](https://mlsanity.com/paper/1904.11045/) (2019)
- [Revisiting IM2GPS in the Deep Learning Era](https://mlsanity.com/paper/1705.04838/) (2017)
- [CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps](https://mlsanity.com/paper/1808.02130/) (2018)
- [Where in the World is this Image? Transformer-based Geo-localization in the Wild](https://mlsanity.com/paper/2204.13861/) (2022)
- [Interpretable Semantic Photo Geolocation](https://mlsanity.com/paper/2104.14995/) (2021)
- [Leveraging EfficientNet and Contrastive Learning for Accurate Global-scale Location Estimation](https://mlsanity.com/paper/2105.07645/) (2021)

## Built on by (most cited first, 30 of 55)

- [GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization](https://mlsanity.com/paper/2309.16020/) (2023)
- [PIGEON: Predicting Image Geolocations](https://mlsanity.com/paper/2307.05845/) (2023)
- [OpenStreetView-5M: The Many Roads to Global Visual Geolocation](https://mlsanity.com/paper/2404.18873/) (2024)
- [G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models](https://mlsanity.com/paper/2405.14702/) (2024)
- [Img2Loc: Revisiting Image Geolocalization using Multi-modality Foundation Models and Image-based Retrieval-Augmented Generation](https://mlsanity.com/paper/2403.19584/) (2024)
- [Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models](https://mlsanity.com/paper/2506.14674/) (2025)
- [GeoReasoner: Geo-localization with Reasoning in Street Views using a Large Vision-Language Model](https://mlsanity.com/paper/2406.18572/) (2024)
- [Geolocation with Real Human Gameplay Data: A Large-Scale Dataset and Human-Like Reasoning Framework](https://mlsanity.com/paper/2502.13759/) (2025)
- [Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation](https://mlsanity.com/paper/2412.06781/) (2024)
- [Granular Privacy Control for Geolocation with Vision Language Models](https://mlsanity.com/paper/2407.04952/) (2024)
- [AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization](https://mlsanity.com/paper/2407.08156/) (2024)
- [GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalization](https://mlsanity.com/paper/2505.13731/) (2025)
- [GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chains](https://mlsanity.com/paper/2505.18700/) (2025)
- [Statewide Visual Geolocalization in the Wild](https://mlsanity.com/paper/2409.16763/) (2024)
- [Evaluating Precise Geolocation Inference Capabilities of Vision Language Models](https://mlsanity.com/paper/2502.14412/) (2025)
- [From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Models](https://mlsanity.com/paper/2508.01608/) (2025)
- [Towards Interactive Global Geolocation Assistant](https://mlsanity.com/paper/2412.08907/) (2024)
- [GT-Loc: Unifying When and Where in Images Through a Joint Embedding Space](https://mlsanity.com/paper/2507.10473/) (2025)
- [CityGuessr: City-Level Video Geo-Localization on a Global Scale](https://mlsanity.com/paper/2411.06344/) (2024)
- [Scaling Image Geo-Localization to Continent Level](https://mlsanity.com/paper/2510.26795/) (2025)
- [GeoLocSFT: Efficient Visual Geolocation via Supervised Fine-Tuning of Multimodal Foundation Models](https://mlsanity.com/paper/2506.01277/) (2025)
- [Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalization](https://mlsanity.com/paper/2601.05432/) (2026)
- [NAVIG: Natural Language-guided Analysis with Vision Language Models for Image Geo-localization](https://mlsanity.com/paper/2502.14638/) (2025)
- [GeoToken: Hierarchical Geolocalization of Images via Next Token Prediction](https://mlsanity.com/paper/2511.01082/) (2025)
- [LocDiff: Identifying Locations on Earth by Diffusing in the Hilbert Space](https://mlsanity.com/paper/2503.18142/) (2025)
- [GAEA: A Geolocation Aware Conversational Assistant](https://mlsanity.com/paper/2503.16423/) (2025)
- [GeoShield: Safeguarding Geolocation Privacy from Vision-Language Models via Adversarial Perturbations](https://mlsanity.com/paper/2508.03209/) (2025)
- [Where on Earth? A Vision-Language Benchmark for Probing Model Geolocation Skills Across Scales](https://mlsanity.com/paper/2510.10880/) (2025)
- [GeoAgent: Learning to Geolocate Everywhere with Reinforced Geographic Characteristics](https://mlsanity.com/paper/2602.12617/) (2026)
- [Scale-adaptive UAV Geo-localization via Height-aware Partition Learning](https://mlsanity.com/paper/2412.11535/) (2024)

## Abstract (arXiv)

Determining the exact latitude and longitude that a photo was taken is a useful and widely applicable task, yet it remains exceptionally difficult despite the accelerated progress of other computer vision tasks. Most previous approaches have opted to learn a single representation of query images, which are then classified at different levels of geographic granularity. These approaches fail to exploit the different visual cues that give context to different hierarchies, such as the country, state, and city level. To this end, we introduce an end-to-end transformer-based architecture that exploits the relationship between different geographic levels (which we refer to as hierarchies) and the corresponding visual scene information in an image through hierarchical cross-attention. We achieve this by learning a query for each geographic hierarchy and scene type. Furthermore, we learn a separate representation for different environmental scenes, as different scenes in the same location are often defined by completely different visual features. We achieve state of the art street level accuracy on 4 standard geo-localization datasets : Im2GPS, Im2GPS3k, YFCC4k, and YFCC26k, as well as qualitatively demonstrate how our method learns different representations for different visual hierarchies and scenes, which has not been demonstrated in the previous methods. These previous testing datasets mostly consist of iconic landmarks or images taken from social media, which makes them either a memorization task, or biased towards certain places. To address this issue we introduce a much harder testing dataset, Google-World-Streets-15k, comprised of images taken from Google Streetview covering the whole planet and present state of the art results. Our code will be made available in the camera-ready version.