# Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks

arXiv: https://arxiv.org/abs/1609.06846 · HTML: https://mlsanity.com/paper/1609.06846/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2016.

## Built on by (most cited first, 25 of 25)

- [Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks](https://mlsanity.com/paper/2107.07933/) (2021)
- [Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks](https://mlsanity.com/paper/2309.16499/) (2023)
- [Attention-based Point Cloud Edge Sampling](https://mlsanity.com/paper/2302.14673/) (2023)
- [On Circuit-based Hybrid Quantum Neural Networks for Remote Sensing Imagery Classification](https://mlsanity.com/paper/2109.09484/) (2021)
- [Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly](https://mlsanity.com/paper/2301.05033/) (2023)
- [Representation Learning for Remote Sensing: An Unsupervised Sensor Fusion Approach](https://mlsanity.com/paper/2108.05094/) (2021)
- [Context-self contrastive pretraining for crop type semantic segmentation](https://mlsanity.com/paper/2104.04310/) (2021)
- [Adaptive Frequency Enhancement Network for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2504.02647/) (2025)
- [An Attention-Fused Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery](https://mlsanity.com/paper/2105.04132/) (2021)
- [DLA-Net: Learning Dual Local Attention Features for Semantic Segmentation of Large-Scale Building Facade Point Clouds](https://mlsanity.com/paper/2106.00376/) (2021)
- [FLAIR-HUB: Large-scale Multimodal Dataset for Land Cover and Crop Mapping](https://mlsanity.com/paper/2506.07080/) (2025)
- [Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications](https://mlsanity.com/paper/2301.01200/) (2022)
- [LMFNet: An Efficient Multimodal Fusion Approach for Semantic Segmentation in High-Resolution Remote Sensing](https://mlsanity.com/paper/2404.13659/) (2024)
- [Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey](https://mlsanity.com/paper/2402.12923/) (2024)
- [Forest Inspection Dataset for Aerial Semantic Segmentation and Depth Estimation](https://mlsanity.com/paper/2403.06621/) (2024)
- [Multi-modal land cover mapping of remote sensing images using pyramid attention and gated fusion networks](https://mlsanity.com/paper/2111.03845/) (2021)
- [SAMBLE: Shape-Specific Point Cloud Sampling for an Optimal Trade-Off Between Local Detail and Global Uniformity](https://mlsanity.com/paper/2504.19581/) (2025)
- [BELDE: Building a Large-scale Earth-observation Land-cover Dataset for Europe](https://mlsanity.com/paper/2606.20909/) (2026)
- [A Histogram Thresholding Improvement to Mask R-CNN for Scalable Segmentation of New and Old Rural Buildings](https://mlsanity.com/paper/2102.04838/) (2021)
- [Spatiotemporal Fusion in Remote Sensing](https://mlsanity.com/paper/2107.02701/) (2021)
- [Deep Learning and Earth Observation to Support the Sustainable Development Goals](https://mlsanity.com/paper/2112.11367/) (2021)
- [GAN-Based Object Removal in High-Resolution Satellite Images](https://mlsanity.com/paper/2301.11726/) (2023)
- [Segment Using Just One Example](https://mlsanity.com/paper/2408.07393/) (2024)
- [Magnifier: A Multi-grained Neural Network-based Architecture for Burned Area Delineation](https://mlsanity.com/paper/2504.19589/) (2025)
- [From Pixels to Images: A Structural Survey of Deep Learning Paradigms in Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2505.15147/) (2025)

## Abstract (arXiv)

This work investigates the use of deep fully convolutional neural networks (DFCNN) for pixel-wise scene labeling of Earth Observation images. Especially, we train a variant of the SegNet architecture on remote sensing data over an urban area and study different strategies for performing accurate semantic segmentation. Our contributions are the following: 1) we transfer efficiently a DFCNN from generic everyday images to remote sensing images; 2) we introduce a multi-kernel convolutional layer for fast aggregation of predictions at multiple scales; 3) we perform data fusion from heterogeneous sensors (optical and laser) using residual correction. Our framework improves state-of-the-art accuracy on the ISPRS Vaihingen 2D Semantic Labeling dataset.