# OmniCity: Omnipotent City Understanding with Multi-level and Multi-view Images

arXiv: https://arxiv.org/abs/2208.00928 · HTML: https://mlsanity.com/paper/2208.00928/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 71 references that are in the map.
- First posted on arXiv 1 Aug 2022.

## Builds on (most cited first, 30 of 51)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019)
- [DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images](https://mlsanity.com/paper/1805.06561/) (2018)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [The ApolloScape Open Dataset for Autonomous Driving and its Application](https://mlsanity.com/paper/1803.06184/) (2018)
- [VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval](https://mlsanity.com/paper/2011.12172/) (2020)
- [Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching](https://mlsanity.com/paper/2005.03860/) (2020)
- [CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification](https://mlsanity.com/paper/1903.09254/) (2019)
- [CARAFE: Content-Aware ReAssembly of FEatures](https://mlsanity.com/paper/1905.02188/) (2019)
- [Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localization](https://mlsanity.com/paper/2103.06818/) (2021)
- [Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation](https://mlsanity.com/paper/1904.06807/) (2019)
- [Bridging the Domain Gap for Ground-to-Aerial Image Matching](https://mlsanity.com/paper/1904.11045/) (2019)
- [Localizing and Orienting Street Views Using Overhead Imagery](https://mlsanity.com/paper/1608.00161/) (2016)
- [Predicting Ground-Level Scene Layout from Aerial Imagery](https://mlsanity.com/paper/1612.02709/) (2016)
- [Rope3D: TheRoadside Perception Dataset for Autonomous Driving and Monocular 3D Object Detection Task](https://mlsanity.com/paper/2203.13608/) (2022)
- [Cross-View Image Matching for Geo-localization in Urban Environments](https://mlsanity.com/paper/1703.07815/) (2017)
- [Cross-View Image Synthesis using Conditional GANs](https://mlsanity.com/paper/1803.03396/) (2018)

## Built on by (most cited first, 21 of 21)

- [CityDreamer: Compositional Generative Model of Unbounded 3D Cities](https://mlsanity.com/paper/2309.00610/) (2023)
- [Where am I? Cross-View Geo-localization with Natural Language Descriptions](https://mlsanity.com/paper/2412.17007/) (2024)
- [Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion](https://mlsanity.com/paper/2401.10786/) (2024)
- [3D Scene Generation: A Survey](https://mlsanity.com/paper/2505.05474/) (2025)
- [CrossViewDiff: A Cross-View Diffusion Model for Satellite-to-Street View Synthesis](https://mlsanity.com/paper/2408.14765/) (2024)
- [SG-BEV: Satellite-Guided BEV Fusion for Cross-View Semantic Segmentation](https://mlsanity.com/paper/2404.02638/) (2024)
- [3D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisions](https://mlsanity.com/paper/2404.04823/) (2024)
- [Leveraging BEV Paradigm for Ground-to-Aerial Image Synthesis](https://mlsanity.com/paper/2408.01812/) (2024)
- [Seeing through Satellite Images at Street Views](https://mlsanity.com/paper/2505.17001/) (2025)
- [UrbanFeel: A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective](https://mlsanity.com/paper/2509.22228/) (2025)
- [MajutsuCity: Language-driven Aesthetic-adaptive City Generation with Controllable 3D Assets and Layouts](https://mlsanity.com/paper/2511.20415/) (2025)
- [Prompt-Driven Building Footprint Extraction in Aerial Images with Offset-Building Model](https://mlsanity.com/paper/2310.16717/) (2023)
- [PolyFootNet: Extracting Polygonal Building Footprints in Off-Nadir Remote Sensing Images](https://mlsanity.com/paper/2408.08645/) (2024)
- [Sat2City v2: Native 3D City Asset Generation from a Single Satellite Image](https://mlsanity.com/paper/2606.24138/) (2026)
- [Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization](https://mlsanity.com/paper/2606.30576/) (2026)
- [DialogueVPR: Towards Conversational Visual Place Recognition](https://mlsanity.com/paper/2607.14115/) (2026)
- [ObliCity: A Benchmark and Baseline for Roof-to-Ground Projection Displacement Correction](https://mlsanity.com/paper/2607.25210/) (2026)
- [Bird Eye-View to Street-View: A Survey](https://mlsanity.com/paper/2405.08961/) (2024)
- [Fine-Grained Building Function Recognition from Street-View Images via Geometry-Aware Semi-Supervised Learning](https://mlsanity.com/paper/2408.09460/) (2024)
- [SatSAM2: Motion-Constrained Video Object Tracking in Satellite Imagery using Promptable SAM2 and Kalman Priors](https://mlsanity.com/paper/2511.18264/) (2025)
- [Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agents](https://mlsanity.com/paper/2609.12533/) (2026)

## Abstract (arXiv)

This paper presents OmniCity, a new dataset for omnipotent city understanding from multi-level and multi-view images. More precisely, the OmniCity contains multi-view satellite images as well as street-level panorama and mono-view images, constituting over 100K pixel-wise annotated images that are well-aligned and collected from 25K geo-locations in New York City. To alleviate the substantial pixel-wise annotation efforts, we propose an efficient street-view image annotation pipeline that leverages the existing label maps of satellite view and the transformation relations between different views (satellite, panorama, and mono-view). With the new OmniCity dataset, we provide benchmarks for a variety of tasks including building footprint extraction, height estimation, and building plane/instance/fine-grained segmentation. Compared with the existing multi-level and multi-view benchmarks, OmniCity contains a larger number of images with richer annotation types and more views, provides more benchmark results of state-of-the-art models, and introduces a novel task for fine-grained building instance segmentation on street-level panorama images. Moreover, OmniCity provides new problem settings for existing tasks, such as cross-view image matching, synthesis, segmentation, detection, etc., and facilitates the developing of new methods for large-scale city understanding, reconstruction, and simulation. The OmniCity dataset as well as the benchmarks will be available at https://city-super.github.io/omnicity.