# Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localization

arXiv: https://arxiv.org/abs/2103.06818 · HTML: https://mlsanity.com/paper/2103.06818/

## Key facts

- Cited by 113 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 11 Mar 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Deep multi-scale video prediction beyond mean square error](https://mlsanity.com/paper/1511.05440/) (2015)
- [Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks](https://mlsanity.com/paper/1604.04382/) (2016)
- [Wide-Area Image Geolocalization with Aerial Reference Imagery](https://mlsanity.com/paper/1510.03743/) (2015)
- [Lending Orientation to Neural Networks for Cross-view Geo-localization](https://mlsanity.com/paper/1903.12351/) (2019)
- [Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching](https://mlsanity.com/paper/2005.03860/) (2020)
- [Bridging the Domain Gap for Ground-to-Aerial Image Matching](https://mlsanity.com/paper/1904.11045/) (2019)
- [Localizing and Orienting Street Views Using Overhead Imagery](https://mlsanity.com/paper/1608.00161/) (2016)
- [Optimal Feature Transport for Cross-View Image Geo-Localization](https://mlsanity.com/paper/1907.05021/) (2019)
- [Predicting Ground-Level Scene Layout from Aerial Imagery](https://mlsanity.com/paper/1612.02709/) (2016)
- [CIAGAN: Conditional Identity Anonymization Generative Adversarial Networks](https://mlsanity.com/paper/2005.09544/) (2020)
- [Cross-View Image Synthesis using Conditional GANs](https://mlsanity.com/paper/1803.03396/) (2018)
- [Semantic Cross-View Matching](https://mlsanity.com/paper/1511.00098/) (2015)
- [Cross-view image synthesis using geometry-guided conditional GANs](https://mlsanity.com/paper/1808.05469/) (2018)
- [Generative Adversarial Frontal View to Bird View Synthesis](https://mlsanity.com/paper/1808.00327/) (2018)
- [Semantic Image Based Geolocation Given a Map](https://mlsanity.com/paper/1609.00278/) (2016)

## Built on by (most cited first, 30 of 113)

- [GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization](https://mlsanity.com/paper/2309.16020/) (2023)
- [TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization](https://mlsanity.com/paper/2204.00097/) (2022)
- [LENS: Localization enhanced by NeRF synthesis](https://mlsanity.com/paper/2110.06558/) (2021)
- [Sample4Geo: Hard Negative Sampling For Cross-View Geo-Localisation](https://mlsanity.com/paper/2303.11851/) (2023)
- [Street-View Image Generation from a Bird's-Eye View Layout](https://mlsanity.com/paper/2301.04634/) (2023)
- [Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image](https://mlsanity.com/paper/2204.04752/) (2022)
- [Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment](https://mlsanity.com/paper/2312.06960/) (2023)
- [Cross-view Geo-localization via Learning Disentangled Geometric Layout Correspondence](https://mlsanity.com/paper/2212.04074/) (2022)
- [Where in the World is this Image? Transformer-based Geo-localization in the Wild](https://mlsanity.com/paper/2204.13861/) (2022)
- [PIGEON: Predicting Image Geolocations](https://mlsanity.com/paper/2307.05845/) (2023)
- [Where We Are and What We're Looking At: Query Based Worldwide Image Geo-localization Using Hierarchies and Scenes](https://mlsanity.com/paper/2303.04249/) (2023)
- [Visual Cross-View Metric Localization with Dense Uncertainty Estimates](https://mlsanity.com/paper/2208.08519/) (2022)
- [Vision-Based UAV Self-Positioning in Low-Altitude Urban Environments](https://mlsanity.com/paper/2201.09201/) (2022)
- [SliceMatch: Geometry-guided Aggregation for Cross-View Pose Estimation](https://mlsanity.com/paper/2211.14651/) (2022)
- [Boosting 3-DoF Ground-to-Satellite Camera Localization Accuracy via Geometry-Guided Cross-View Transformer](https://mlsanity.com/paper/2307.08015/) (2023)
- [Accurate 3-DoF Camera Geo-Localization via Ground-to-Satellite Image Matching](https://mlsanity.com/paper/2203.14148/) (2022)
- [Cross-view image geo-localization with Panorama-BEV Co-Retrieval Network](https://mlsanity.com/paper/2408.05475/) (2024)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [Simple, Effective and General: A New Backbone for Cross-view Image Geo-localization](https://mlsanity.com/paper/2302.01572/) (2023)
- [Convolutional Cross-View Pose Estimation](https://mlsanity.com/paper/2303.05915/) (2023)
- [Cross-View Image Sequence Geo-localization](https://mlsanity.com/paper/2210.14295/) (2022)
- [Unleashing Unlabeled Data: A Paradigm for Cross-View Geo-Localization](https://mlsanity.com/paper/2403.14198/) (2024)
- [GeoDTR+: Toward generic cross-view geolocalization via geometric disentanglement](https://mlsanity.com/paper/2308.09624/) (2023)
- [ConGeo: Robust Cross-view Geo-localization across Ground View Variations](https://mlsanity.com/paper/2403.13965/) (2024)
- [CVLNet: Cross-View Semantic Correspondence Learning for Video-based Camera Localization](https://mlsanity.com/paper/2208.03660/) (2022)
- [Learning Dense Flow Field for Highly-accurate Cross-view Camera Localization](https://mlsanity.com/paper/2309.15556/) (2023)
- [Game4Loc: A UAV Geo-Localization Benchmark from Game Data](https://mlsanity.com/paper/2409.16925/) (2024)
- [GAMa: Cross-view Video Geo-localization](https://mlsanity.com/paper/2207.02431/) (2022)
- [CrossViewDiff: A Cross-View Diffusion Model for Satellite-to-Street View Synthesis](https://mlsanity.com/paper/2408.14765/) (2024)
- [CV-Cities: Advancing Cross-View Geo-Localization in Global Cities](https://mlsanity.com/paper/2411.12431/) (2024)

## Abstract (arXiv)

The goal of cross-view image based geo-localization is to determine the location of a given street view image by matching it against a collection of geo-tagged satellite images. This task is notoriously challenging due to the drastic viewpoint and appearance differences between the two domains. We show that we can address this discrepancy explicitly by learning to synthesize realistic street views from satellite inputs. Following this observation, we propose a novel multi-task architecture in which image synthesis and retrieval are considered jointly. The rationale behind this is that we can bias our network to learn latent feature representations that are useful for retrieval if we utilize them to generate images across the two input domains. To the best of our knowledge, ours is the first approach that creates realistic street views from satellite images and localizes the corresponding query street-view simultaneously in an end-to-end manner. In our experiments, we obtain state-of-the-art performance on the CVUSA and CVACT benchmarks. Finally, we show compelling qualitative results for satellite-to-street view synthesis.