mlsanityData to 14 Oct 2026

From Pixels to Images: A Structural Survey of Deep Learning Paradigms in Remote Sensing Image Semantic Segmentation

arXiv 2505.15147 ↗First posted 21 May 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Remote sensing images (RSIs) capture both natural and human-induced changes on the Earth's surface. Semantic segmentation (SS) of RSIs enables the fine-grained interpretation of surface features, making it a critical task in RS analysis. With the increasing diversity and volume of RSIs collected by sensors on various platforms, traditional processing methods struggle to maintain efficiency and accuracy. In response, deep learning (DL) has emerged as a transformative approach, enabling substantial advances in remote sensing image semantic segmentation (RSISS). As researchers continue to explore end-to-end SS, DL-based RSISS has undergone a structural evolution from pixel-level and patch-based classification to tile-level and image-level segmentation. However, existing reviews often focus on individual components, such as supervision strategies or fusion stages, and lack a unified operational perspective aligned with segmentation granularity and the training/inference pipeline. This paper provides a comprehensive review by organizing DL-based RSISS into a pixel-patch-tile-image hierarchy, covering early pixel-based methods, prevailing patch-based and tile-based techniques, and emerging image-based approaches. Specifically, the survey analyzes four supervision strategies, eleven feature extraction strategies, and six information fusion strategies, revealing the field's progression from local to global feature extraction, from traditional DL architectures to foundation models, and from unimodal to multimodal segmentation. This review offers a holistic and structured understanding of DL-based RSISS, highlighting representative datasets, comparative insights, and open challenges related to data scale, model efficiency, domain robustness, and multimodal integration. Furthermore, to facilitate reproducible research, curated code collections are provided at: https://github.com/quanweiliu/RSISS.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 89 of its 379 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

89 of 379 references.

References · 89

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2023Segment Anythingcited by 8,234
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  8. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,380
  9. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
  10. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  11. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 408
  12. 2017Road Extraction by Deep Residual U-Netcited by 333
38 more
  1. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 300
  2. 2018DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Imagescited by 277
  3. 2019ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed datacited by 261
  4. 2021LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentationcited by 261
  5. 2017LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentationcited by 214
  6. 2018DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptationcited by 214
  7. 2020FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understandingcited by 181
  8. 2021UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagerycited by 167
  9. 2020More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classificationcited by 126
  10. 2018Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Modelscited by 119
  11. 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
  12. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 73
  13. 2017Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networkscited by 64
  14. 2022Multimodal Fusion Transformer for Remote Sensing Image Classificationcited by 64
  15. 2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
  16. 2016Going Deeper with Contextual CNN for Hyperspectral Image Classificationcited by 59
  17. 2023RRSIS: Referring Remote Sensing Image Segmentationcited by 59
  18. 2023Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networkscited by 55
  19. 2022Towards Good Practices for Missing Modality Robust Action Recognitioncited by 54
  20. 2019Deep Learning for Classification of Hyperspectral Data: A Comparative Reviewcited by 51
  21. 2021Hyperspectral Image Classification-Traditional to Deep Models: A Survey for Future Prospectscited by 49
  22. 2025MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classificationcited by 44
  23. 2021Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Modelcited by 43
  24. 2025SegEarth-R1: Geospatial Pixel Reasoning via Large Language Modelcited by 42
  25. 2024SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentationcited by 41
  26. 2020LandCover.ai: Dataset for Automatic Mapping of Buildings, Woodlands, Water and Roads from Aerial Imagerycited by 40
  27. 2017Algorithms for Semantic Segmentation of Multispectral Remote Sensing Imagery using Deep Learningcited by 37
  28. 2016Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagerycited by 36
  29. 2020Feature Extraction for Hyperspectral Imagery: The Evolution from Shallow to Deep (Overview and Toolbox)cited by 36
  30. 2018Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Datacited by 35
  31. 2019Unsupervised Domain Adaptation using Generative Adversarial Networks for Semantic Segmentation of Aerial Imagescited by 32
  32. 2021Global and Local Contrastive Self-Supervised Learning for Semantic Segmentation of HR Remote Sensing Imagescited by 31
  33. 2021Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Imagescited by 28
  34. 2024CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentationcited by 27
  35. 2019ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networkscited by 26
  36. 2020Multi-stage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Imagescited by 26
  37. 2024Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentationcited by 26
  38. 2016Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networkscited by 25

50 most cited of 89 All 89 (JSON)

Built on by

Not cited yet.