mlsanityData to 7 Oct 2026

Parameter-Efficient Modality-Balanced Symmetric Fusion for Multimodal Remote Sensing Semantic Segmentation

arXiv 2603.17705 ↗First posted 18 Mar 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Multimodal remote sensing semantic segmentation enhances scene interpretation by exploiting complementary physical cues from heterogeneous data. Although pretrained Vision Foundation Models (VFMs) provide strong general-purpose representations, adapting them to multimodal tasks often incurs substantial computational overhead and is prone to modality imbalance, where the contribution of auxiliary modalities is suppressed during optimization. To address these challenges, we propose MoBaNet, a parameter-efficient and modality-balanced symmetric fusion framework. Built upon a largely frozen VFM backbone, MoBaNet adopts a symmetric dual-stream architecture to preserve generalizable representations while minimizing the number of trainable parameters. Specifically, we design a Cross-modal Prompt-Injected Adapter (CPIA) to enable deep semantic interaction by generating shared prompts and injecting them into bottleneck adapters under the frozen backbone. To obtain compact and discriminative multimodal representations for decoding, we further introduce a Difference-Guided Gated Fusion Module (DGFM), which adaptively fuses paired stage features by explicitly leveraging cross-modal discrepancy to guide feature selection. Furthermore, we propose a Modality-Conditional Random Masking (MCRM) strategy to mitigate modality imbalance by masking one modality only during training and imposing hard-pixel auxiliary supervision on modality-specific branches. Extensive experiments on the ISPRS Vaihingen and Potsdam benchmarks demonstrate that MoBaNet achieves state-of-the-art performance with significantly fewer trainable parameters than full fine-tuning, validating its effectiveness for robust and balanced multimodal fusion. The source code in this work is available at https://github.com/sauryeo/MoBaNet.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 55 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

30 of 55 references.

References · 30

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2016Pyramid Scene Parsing Networkcited by 2,472
  8. 2022Visual Prompt Tuningcited by 1,674
  9. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  10. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  11. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  12. 2019ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed datacited by 260
18 more
  1. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 202
  2. 2021UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagerycited by 166
  3. 2020More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classificationcited by 126
  4. 2022Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Reviewcited by 73
  5. 2017Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networkscited by 64
  6. 2021A Novel Transformer Based Semantic Segmentation Scheme for Fine-Resolution Remote Sensing Imagescited by 55
  7. 2021Crop mapping from image time series: deep learning with multi-scale label hierarchiescited by 42
  8. 2022Self-supervised Vision Transformers for Joint SAR-optical Representation Learningcited by 40
  9. 2020Multi-stage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Imagescited by 26
  10. 2018Semantic Labeling in Very High Resolution Images via a Self-Cascaded Convolutional Neural Networkcited by 20
  11. 2016Dense semantic labeling of sub-decimeter resolution images with convolutional neural networkscited by 19
  12. 2025Improving Multimodal Learning via Imbalanced Learningcited by 16
  13. 2024A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentationcited by 15
  14. 2016High-Resolution Semantic Labeling with Convolutional Neural Networkscited by 11
  15. 2025Reducing Unimodal Bias in Multi-Modal Semantic Segmentation with Multi-Scale Functional Entropy Regularizationcited by 9
  16. 2021An Attention-Fused Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagerycited by 7
  17. 2024Adapting Vision Foundation Models for Robust Cloud Segmentation in Remote Sensing Imagescited by 7
  18. 2023Imbalance Knowledge-Driven Multi-modal Network for Land-Cover Semantic Segmentation Using Images and LiDAR Point Cloudscited by 4

Built on by

Not cited yet.