# Understanding urban landuse from the above and ground perspectives: a deep learning, multimodal solution

arXiv: https://arxiv.org/abs/1905.01752 · HTML: https://mlsanity.com/paper/1905.01752/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2019.

## Built on by (most cited first, 26 of 26)

- [Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review](https://mlsanity.com/paper/2205.01380/) (2022)
- [Enabling Country-Scale Land Cover Mapping with Meter-Resolution Satellite Imagery](https://mlsanity.com/paper/2209.00727/) (2022)
- [Towards a Collective Agenda on AI for Earth Science Data Analysis](https://mlsanity.com/paper/2104.05107/) (2021)
- [OmniCity: Omnipotent City Understanding with Multi-level and Multi-view Images](https://mlsanity.com/paper/2208.00928/) (2022)
- [SG-BEV: Satellite-Guided BEV Fusion for Cross-View Semantic Segmentation](https://mlsanity.com/paper/2404.02638/) (2024)
- [Revisiting Near/Remote Sensing with Geospatial Attention](https://mlsanity.com/paper/2204.01807/) (2022)
- [Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications](https://mlsanity.com/paper/2301.01200/) (2022)
- [Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation](https://mlsanity.com/paper/2407.15512/) (2024)
- [Sensing accident-prone features in urban scenes for proactive driving and accident prevention](https://mlsanity.com/paper/2202.12788/) (2022)
- [Dual-Branch Subpixel-Guided Network for Hyperspectral Image Classification](https://mlsanity.com/paper/2412.03893/) (2024)
- [Impact Assessment of Missing Data in Model Predictions for Earth Observation Applications](https://mlsanity.com/paper/2403.14297/) (2024)
- [Multisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification](https://mlsanity.com/paper/2412.03897/) (2024)
- [Urban land-use analysis using proximate sensing imagery: a survey](https://mlsanity.com/paper/2101.04827/) (2021)
- [Cross-Modal Pre-Aligned Method with Global and Local Information for Remote-Sensing Image and Text Retrieval](https://mlsanity.com/paper/2411.14704/) (2024)
- [Learning to Match Unpaired Data with Minimum Entropy Coupling](https://mlsanity.com/paper/2503.08501/) (2025)
- [Facing the Void: Overcoming Missing Data in Multi-View Imagery](https://mlsanity.com/paper/2205.10592/) (2022)
- [Credible Remote Sensing Scene Classification Using Evidential Fusion on Aerial-Ground Dual-view Images](https://mlsanity.com/paper/2301.00622/) (2023)
- [Jointly Learning Representations for Map Entities via Heterogeneous Graph Contrastive Learning](https://mlsanity.com/paper/2402.06135/) (2024)
- [Unsupervised Urban Land Use Mapping with Street View Contrastive Clustering and a Geographical Prior](https://mlsanity.com/paper/2504.17551/) (2025)
- [Cross-View Localization via Redundant Sliced Observations and A-Contrario Validation](https://mlsanity.com/paper/2508.05369/) (2025)
- [Mapping Vulnerable Populations with AI](https://mlsanity.com/paper/2107.14123/) (2021)
- [Cross-Modal Learning of Housing Quality in Amsterdam](https://mlsanity.com/paper/2403.08915/) (2024)
- [Fine-Grained Building Function Recognition from Street-View Images via Geometry-Aware Semi-Supervised Learning](https://mlsanity.com/paper/2408.09460/) (2024)
- [On What Depends the Robustness of Multi-source Models to Missing Data in Earth Observation?](https://mlsanity.com/paper/2503.19719/) (2025)
- [An Evidence Hierarchy for Bayesian Object Classification via OSINT-Aided Heterogeneous Sensor Fusion](https://mlsanity.com/paper/2605.22259/) (2026)
- [Multi-Modal Building Inspection via Perceiver IO Fusion of Satellite and Street-Level Imagery](https://mlsanity.com/paper/2605.26381/) (2026)

## Abstract (arXiv)

Landuse characterization is important for urban planning. It is traditionally performed with field surveys or manual photo interpretation, two practices that are time-consuming and labor-intensive. Therefore, we aim to automate landuse mapping at the urban-object level with a deep learning approach based on data from multiple sources (or modalities). We consider two image modalities: overhead imagery from Google Maps and ensembles of ground-based pictures (side-views) per urban-object from Google Street View (GSV). These modalities bring complementary visual information pertaining to the urban-objects. We propose an end-to-end trainable model, which uses OpenStreetMap annotations as labels. The model can accommodate a variable number of GSV pictures for the ground-based branch and can also function in the absence of ground pictures at prediction time. We test the effectiveness of our model over the area of Ile-de-France, France, and test its generalization abilities on a set of urban-objects from the city of Nantes, France. Our proposed multimodal Convolutional Neural Network achieves considerably higher accuracies than methods that use a single image modality, making it suitable for automatic landuse map updates. Additionally, our approach could be easily scaled to multiple cities, because it is based on data sources available for many cities worldwide.