mlsanityData to 7 Oct 2026

Boundary-Aligned Contribution Routing for Robust Optical--SAR Object Detection

arXiv 2608.15261 ↗First posted 15 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Optical imagery provides rich appearance cues, whereas synthetic aperture radar (SAR) offers observations that are less sensitive to illumination and weather, making optical--SAR fusion attractive for remote-sensing object detection. However, the presence of multiple modalities does not guarantee beneficial fusion: imperfect spatial, temporal, and semantic correspondence can make an otherwise intact stream conditionally harmful and induce negative cross-modal transfer. We handle this issue through a model-specific task-utility perspective and learn task-conditioned contribution routing using detection supervision alone. The proposed fusion-boundary-aligned routing regulates each modality's contribution before the first learned cross-modal feature-value mixing operation. For architectures with frequent shallow interaction, a Feature Router performs cross-conditioned, group-addressable modulation near the input; for dual-backbone architectures, a Dual-Statistic Semantic Router predicts stream-level contribution weights from modality-specific average and maximum statistics before late semantic fusion. The routers require no explicit utility supervision, quality labels, reconstruction, or distillation. Experiments on M4-SAR and SpaceNet6-OTD cover nominal full inputs, controlled correspondence shifts, missing modalities, and four nonzero modality-corruption scenarios. Across the reported clean-training controls, routing improves full-input mAP50 by 0.5--5.9 points. Relative to the corresponding modality-dropout baselines, it raises missing-modality mAP50 by 7.6--41.6 points and reduces the negative-transfer rate by up to 12.7 percentage points. Spearman correlations between the learned routing weights and model-specific leave-one-modality-out utility range from 0.45 to 0.66, supporting the task-utility interpretation of the routing coefficients.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 68 references that are in the map. First posted on arXiv 15 Aug 2026.

Builds on

34 of 68 references.

References · 34

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2017Squeeze-and-Excitation Networkscited by 4,763
  3. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  4. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  5. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  6. 2021Dynamic Neural Networks: A Surveycited by 405
  7. 2022Multimodal Learning with Transformers: A Surveycited by 399
  8. 2023Multimodal Deep Learningcited by 257
  9. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 183
  10. 2015Cross Modal Distillation for Supervision Transfercited by 173
  11. 2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 167
  12. 2020Channel-wise Knowledge Distillation for Dense Predictioncited by 161
22 more
  1. 2023Multi-modal Learning with Missing Modality via Shared-Specific Feature Modellingcited by 159
  2. 2016Semantic Image Inpainting with Deep Generative Modelscited by 156
  3. 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 94
  4. 2023ICAFusion: Iterative Cross-Attention Guided Feature Fusion for Multispectral Object Detectioncited by 76
  5. 2022Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Reviewcited by 73
  6. 2024Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Expertscited by 69
  7. 2023Cross Modal Transformer: Towards Fast and Robust 3D Object Detectioncited by 68
  8. 2015Multimodal Deep Learning for Robust RGB-D Object Recognitioncited by 56
  9. 2023M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalitiescited by 56
  10. 2020SpaceNet 6: Multi-Sensor All Weather Mapping Datasetcited by 47
  11. 2022SuperYOLO: Super Resolution Assisted Object Detection in Multimodal Remote Sensing Imagerycited by 46
  12. 2021Cross-Modality Attentive Feature Fusion for Object Detection in Multispectral Remote Sensing Imagerycited by 44
  13. 2021Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directionscited by 35
  14. 2025SimMLM: A Simple Framework for Multi-modal Learning with Missing Modalitycited by 22
  15. 2023Calibrating Multimodal Learningcited by 17
  16. 2021Feature-enhanced Generation and Multi-modality Fusion based Deep Neural Network for Brain Tumor Segmentation with Missing MR Modalitiescited by 13
  17. 2024MaDiNet: Mamba Diffusion Network for SAR Target Detectioncited by 13
  18. 2025Resilient Sensor Fusion under Adverse Sensor Failures via Multi-Modal Expert Fusioncited by 13
  19. 2023Reliable Representation Learning for Incomplete Multi-View Missing Multi-Label Classificationcited by 12
  20. 2025M4-SAR: A Multi-Resolution, Multi-Polarization, Multi-Scene, Multi-Source Dataset and Benchmark for optical-SAR Object Detectioncited by 8
  21. 2024Multi-modal Learning with Missing Modality in Predicting Axillary Lymph Node Metastasiscited by 3
  22. 2025Towards Robust Optical-SAR Object Detection under Missing Modalities: A Dynamic Quality-Aware Fusion Frameworkcited by 2

Built on by

Not cited yet.