# Multi-modal land cover mapping of remote sensing images using pyramid attention and gated fusion networks

arXiv: https://arxiv.org/abs/2111.03845 · HTML: https://mlsanity.com/paper/2111.03845/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 63 references that are in the map.
- First posted on arXiv 6 Nov 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification](https://mlsanity.com/paper/2008.05457/) (2020)
- [Efficient piecewise training of deep structured models for semantic segmentation](https://mlsanity.com/paper/1504.01013/) (2015)
- [Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks](https://mlsanity.com/paper/1711.08681/) (2017)
- [Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysis](https://mlsanity.com/paper/2001.01306/) (2020)
- [Indoor Semantic Segmentation using depth information](https://mlsanity.com/paper/1301.3572/) (2013)
- [Deep Learning for Classification of Hyperspectral Data: A Comparative Review](https://mlsanity.com/paper/1904.10674/) (2019)
- [Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection](https://mlsanity.com/paper/1612.01337/) (2016)
- [Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery](https://mlsanity.com/paper/1606.02585/) (2016)
- [Semantic Segmentation of Earth Observation Data Using Multimodal and Multi-scale Deep Networks](https://mlsanity.com/paper/1609.06846/) (2016)
- [Dense Dilated Convolutions Merging Network for Land Cover Classification](https://mlsanity.com/paper/2003.04027/) (2020)
- [Urban Land Cover Classification with Missing Data Modalities Using Deep Convolutional Neural Networks](https://mlsanity.com/paper/1709.07383/) (2017)
- [Joint Learning from Earth Observation and OpenStreetMap Data to Get Faster Better Semantic Maps](https://mlsanity.com/paper/1705.06057/) (2017)
- [The 1st Agriculture-Vision Challenge: Methods and Results](https://mlsanity.com/paper/2004.09754/) (2020)
- [Multi-view Self-Constructing Graph Convolutional Networks with Adaptive Class Weighting Loss for Semantic Segmentation](https://mlsanity.com/paper/2004.10327/) (2020)
- [Self-Constructing Graph Convolutional Networks for Semantic Labeling](https://mlsanity.com/paper/2003.06932/) (2020)

## Built on by (most cited first, 1 of 1)

- [Dual Atrous Separable Convolution for Improving Agricultural Semantic Segmentation](https://mlsanity.com/paper/2506.22570/) (2025)

## Abstract (arXiv)

Multi-modality data is becoming readily available in remote sensing (RS) and can provide complementary information about the Earth's surface. Effective fusion of multi-modal information is thus important for various applications in RS, but also very challenging due to large domain differences, noise, and redundancies. There is a lack of effective and scalable fusion techniques for bridging multiple modality encoders and fully exploiting complementary information. To this end, we propose a new multi-modality network (MultiModNet) for land cover mapping of multi-modal remote sensing data based on a novel pyramid attention fusion (PAF) module and a gated fusion unit (GFU). The PAF module is designed to efficiently obtain rich fine-grained contextual representations from each modality with a built-in cross-level and cross-view attention fusion mechanism, and the GFU module utilizes a novel gating mechanism for early merging of features, thereby diminishing hidden redundancies and noise. This enables supplementary modalities to effectively extract the most valuable and complementary information for late feature fusion. Extensive experiments on two representative RS benchmark datasets demonstrate the effectiveness, robustness, and superiority of the MultiModNet for multi-modal land cover classification.