mlsanityData to 14 Oct 2026

Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review

arXiv 2205.01380 ↗First posted 3 May 2022cs.CV cs.LG eess.SPcited by 73

Not on the map. Why?

Abstract (arXiv)

With the extremely rapid advances in remote sensing (RS) technology, a great quantity of Earth observation (EO) data featuring considerable and complicated heterogeneity is readily available nowadays, which renders researchers an opportunity to tackle current geoscience applications in a fresh way. With the joint utilization of EO data, much research on multimodal RS data fusion has made tremendous progress in recent years, yet these developed traditional algorithms inevitably meet the performance bottleneck due to the lack of the ability to comprehensively analyse and interpret these strongly heterogeneous data. Hence, this non-negligible limitation further arouses an intense demand for an alternative tool with powerful processing competence. Deep learning (DL), as a cutting-edge technology, has witnessed remarkable breakthroughs in numerous computer vision tasks owing to its impressive ability in data representation and reconstruction. Naturally, it has been successfully applied to the field of multimodal RS data fusion, yielding great improvement compared with traditional methods. This survey aims to present a systematic overview in DL-based multimodal RS data fusion. More specifically, some essential knowledge about this topic is first given. Subsequently, a literature survey is conducted to analyse the trends of this field. Some prevalent sub-fields in the multimodal RS data fusion are then reviewed in terms of the to-be-fused data modalities, i.e., spatiospectral, spatiotemporal, light detection and ranging-optical, synthetic aperture radar-optical, and RS-Geospatial Big Data fusion. Furthermore, We collect and summarize some valuable resources for the sake of the development in multimodal RS data fusion. Finally, the remaining challenges and potential future directions are highlighted.

Key facts. Cited by 73 papers in the map (data through 14 Oct 2026). Builds on 37 of its 216 references that are in the map. First posted on arXiv 3 May 2022.

Builds on

37 of 216 references.

References · 37

  1. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  2. 2017Deep learning in remote sensing: a reviewcited by 128
  3. 2020More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classificationcited by 126
  4. 2020Graph Convolutional Networks for Hyperspectral Image Classificationcited by 113
  5. 2015Hyperspectral pansharpening: a reviewcited by 55
  6. 2017A Multi-Scale and Multi-Depth Convolutional Neural Network for Remote Sensing Imagery Pan-Sharpeningcited by 54
  7. 2021Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Modelcited by 43
  8. 2020Feature Extraction for Hyperspectral Imagery: The Evolution from Shallow to Deep (Overview and Toolbox)cited by 36
  9. 2019Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Netcited by 35
  10. 2021Deep Learning for UAV-based Object Detection and Tracking: A Surveycited by 33
  11. 2022HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpeningcited by 32
  12. 2021Fusformer: A Transformer-based Fusion Approach for Hyperspectral Image Super-resolutioncited by 31
25 more
  1. 2018Pansharpening via Detail Injection Based Convolutional Neural Networkscited by 29
  2. 2020Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolutioncited by 29
  3. 2018Unsupervised Sparse Dirichlet-Net for Hyperspectral Image Super-Resolutioncited by 26
  4. 2019Understanding urban landuse from the above and ground perspectives: a deep learning, multimodal solutioncited by 26
  5. 2017Remote Sensing Image Fusion Based on Two-stream Fusion Networkcited by 25
  6. 2020Classification of Hyperspectral and LiDAR Data Using Coupled CNNscited by 25
  7. 2017Boosting the accuracy of multi-spectral image pan-sharpening by learning a deep residual networkcited by 23
  8. 2020Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusioncited by 21
  9. 2017Target-adaptive CNN-based pansharpeningcited by 20
  10. 2020Guided Deep Decoder: Unsupervised Image Pair Fusioncited by 20
  11. 2022PanFormer: a Transformer Based Model for Pan-sharpeningcited by 18
  12. 2020Coupled Convolutional Neural Network with Adaptive Response Function Learning for Unsupervised Hyperspectral Super-Resolutioncited by 15
  13. 2017Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Networkcited by 14
  14. 2019Unsupervised and Unregistered Hyperspectral Image Super-Resolution with Mutual Dirichlet-Netcited by 13
  15. 2020Hyperspectral Image Super-resolution via Deep Progressive Zero-centric Residual Learningcited by 13
  16. 2021Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstructioncited by 12
  17. 2017Towards seamless multi-view scene analysis from satellite to street-levelcited by 10
  18. 2020Hyperspectral Image Super-Resolution via Deep Prior Regularization with Parameter Estimationcited by 10
  19. 2021Model Inspired Autoencoder for Unsupervised Hyperspectral Image Super-Resolutioncited by 10
  20. 2020PGMAN: An Unsupervised Generative Multi-adversarial Network for Pan-sharpeningcited by 9
  21. 2018Multisource and Multitemporal Data Fusion in Remote Sensingcited by 8
  22. 2018Spatial-Spectral Fusion by Combining Deep Learning and Variation Modelcited by 6
  23. 2020Rethinking CNN-Based Pansharpening: Guided Colorization of Panchromatic Images via GANscited by 6
  24. 2021Aerial Images Meet Crowdsourced Trajectories: A New Approach to Robust Road Extractioncited by 6
  25. 2021A Triple-Double Convolutional Neural Network for Panchromatic Sharpeningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 14; 2024: 21; 2025: 19; 2026: 16.

73 papers.

Sort:

By area:

  1. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  2. 2025TerraMind: Large-Scale Generative Multimodality for Earth Observationcited by 104
  3. 2024Mission Critical -- Satellite Data is a Distinct Modality in Machine Learningcited by 77
  4. 2024OmniSat: Self-Supervised Modality Fusion for Earth Observationcited by 63
  5. 2023Generate Your Own Scotland: Satellite Image Generation Conditioned on Mapscited by 30
  6. 2024Unsupervised Hyperspectral Image Super-Resolution via Self-Supervised Modality Decouplingcited by 26
  7. 2023Better, Not Just More: Data-Centric Machine Learning for Earth Observationcited by 19
  8. 2023Multi-modal learning for geospatial vegetation forecastingcited by 16
  9. 2024A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentationcited by 15
  10. 2024Conditional Controllable Image Fusioncited by 13
  11. 2024ASANet: Asymmetric Semantic Aligning Network for RGB and SAR image land cover classificationcited by 11
  12. 2023The Potential of Visual ChatGPT For Remote Sensingcited by 10
38 more
  1. 2023Cross-Scope Spatial-Spectral Information Aggregation for Hyperspectral Image Super-Resolutioncited by 10
  2. 2022Tensor Decompositions for Hyperspectral Data Processing in Remote Sensing: A Comprehensive Reviewcited by 9
  3. 2023A Spectral Diffusion Prior for Hyperspectral Image Super-Resolutioncited by 8
  4. 2024Adaptive Fusion of Multi-view Remote Sensing data for Optimal Sub-field Crop Yield Predictioncited by 8
  5. 2024Babel: A Scalable Pre-trained Model for Multi-Modal Sensing via Expandable Modality Alignmentcited by 8
  6. 2025Visual Question Answering on Multiple Remote Sensing Image Modalitiescited by 6
  7. 2025FLAIR-HUB: Large-scale Multimodal Dataset for Land Cover and Crop Mappingcited by 6
  8. 2022Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applicationscited by 5
  9. 2024LMFNet: An Efficient Multimodal Fusion Approach for Semantic Segmentation in High-Resolution Remote Sensingcited by 4
  10. 2024Multimodal Deep Learning for Low-Resource Settings: A Vector Embedding Alignment Approach for Healthcare Applicationscited by 4
  11. 2025Rainy: Unlocking Satellite Calibration for Deep Learning in Precipitationcited by 4
  12. 2024Cross-sensor self-supervised training and alignment for remote sensingcited by 3
  13. 2024CarbonSense: A Multimodal Dataset and Baseline for Carbon Flux Modellingcited by 3
  14. 2024Set-CLIP: Exploring Aligned Semantic From Low-Alignment Multimodal Data Through A Distribution Viewcited by 3
  15. 2024Unsupervised Hyperspectral and Multispectral Image Blind Fusion Based on Deep Tucker Decomposition Network with Spatial-Spectral Manifold Learningcited by 3
  16. 2025The Moon's Many Faces: A Single Unified Transformer for Multimodal Lunar Reconstructioncited by 3
  17. 2023Explaining Multimodal Data Fusion: Occlusion Analysis for Wilderness Mappingcited by 2
  18. 2024Multi-Sensor Deep Learning for Glacier Mappingcited by 2
  19. 2025A Decade of Deep Learning for Remote Sensing Spatiotemporal Fusion: Advances, Challenges, and Opportunitiescited by 2
  20. 2025COP-GEN-Beta: Unified Generative Modelling of COPernicus Imagery Thumbnailscited by 2
  21. 2025A comprehensive review of remote sensing in wetland classification and mappingcited by 2
  22. 2025PAD: Phase-Amplitude Decoupling Fusion for Multi-Modal Land Cover Classificationcited by 2
  23. 2026Feature Entanglement-based Quantum Multimodal Fusion Neural Networkcited by 2
  24. 2023Multimodal Hyperspectral Image Classification via Interconnected Fusioncited by 1
  25. 2023Multimodal Object Detection in Remote Sensingcited by 1
  26. 2023A Benchmarking Protocol for SAR Colorization: From Regression to Deep Learning Approachescited by 1
  27. 2024Fusing Climate Data Products using a Spatially Varying Autoencodercited by 1
  28. 2024Data-driven Modeling in Metrology -- A Short Introduction, Current Developments and Future Perspectivescited by 1
  29. 2024Interactive Masked Image Modeling for Multimodal Object Detection in Remote Sensingcited by 1
  30. 2025HyperPointFormer: Multimodal Fusion in 3D Space with Dual-Branch Cross-Attention Transformerscited by 1
  31. 2025Generative AI as a Pillar for Predicting 2D and 3D Wildfire Spread: Beyond Physics-Based Models and Traditional Deep Learningcited by 1
  32. 2025Assessing the Effectiveness of Deep Embeddings for Tree Species Classification in the Dutch Forest Inventorycited by 1
  33. 2025Multimodal Feature Fusion Network with Text Difference Enhancement for Remote Sensing Change Detectioncited by 1
  34. 2025Balanced Diffusion-Guided Fusion for Multimodal Remote Sensing Classificationcited by 1
  35. 2025RAH-VLA: Resolution-Adaptive Hierarchical Vision-Language Alignment for Multimodal Remote Sensing Understandingcited by 1
  36. 2026A unified multimodal understanding and generation model for cross-disciplinary scientific researchcited by 1
  37. 2026FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scalescited by 1
  38. 2023Scalable and Robust Tensor Ring Decomposition for Large-scale Datacited by 0

50 most cited of 73 All 73 (JSON)